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OipLAPTER I 

STATISTICS AS A SCIENCE: AXIOMS OF 
PROBABILITY 

1. Introductory. The word “ statistics ” is defined in 
the Concise Oxford Dictionary as follows : in the plural, 
“ numerical facts systematically collected, as statistics of 
population, crime ; in the singular, “ science of collecting, 
classifying and using statistics.’’ This definition adequately 
conveys the present meaning of the word ; but the term 
was once restricted, as its derivation shows, to systematic 
collections of data descriptive of political communities, a 
domain partly taken over now by the more special word 
demography.” 

The word statistics (in the plural) is used nowadays 
to characterize “ numerical facts systematically collected ” 
in any field whatever of observation or experiment. 
The technique of collecting data and the principles 
to be heeded in order to avoid bias in the interpretation 
are described at length and exemplified in chapters of more 
extensive treatises which the reader may consult. He may 
also form a general idea of practical details by studying 
the prefatory description of method in some actual published 
investigation, for example into housing and economic 
conditions in a particular town or area. In any case th^ 
principles to be observed in arranging a statistical investi- 
gation can be thoroughly grasped only when the analysis 
used to interpret the data is well understood ; and this 
involves a knowledge of the science of statistics (in the 
singular). 

The intermediate stage of tabulation^ by which collected 

A 
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data are set out in the most perspicuous form for analysis 
or inspection with a particular aim, is also usually the 
subject of a chapter, with illustrative examples and 
criticisms, in larger treatises than the present one. Here 
again the reader may learn much from the attentive 
perusal of statistical year-books and similar publications, 
and from the results tabulated in other published investiga¬ 
tions. The principles are those of logical classification of 
different categories; and the art of tabulation rests in 
making the relation of the categories and the numbers 
in various categories as clear as possible to the eye yet 
compact on the printed page. Thus one may have 
statistics of employed persons according to age, sex, 
district, trade and wage ; how can the respective numbers 
best be set out in one or more tables with rows and columns, 
row-totals, column-totals, sub-totals and grand totals? 
This is a typical problem of tabulation, and the chief aids 
towards resolving it rest on experience and common sense. 

Statistics involves classification by number in categories. 
Let us note for further reference the possible relations of 
individuals in two categories A and B. It may be that an 
individual of the collection cannot be both A and B at the 
same time ; for example if a coin falls “ heads,” it certainly 
has not fallen “tails.” The categories A and B are then 
mutually exclusive ; their relation is that of “ either ... 
or.” On the other hand, the categories A and B may be 
of such a kind that an individual may belong to both at 
the same time ; the relation of such categories is that of 
“ both . . . and.” 

^ 2. Statistics as a Science. The concern of the 

present book will for the most part be with statistics (in 
the singular) as a science. The t3^ical order of develop¬ 
ment of the ” exact ” sciences (as they are somewhat 
loosely called) has been along the following lines. First 
of all, the examination of data collected in a particular 
field of inquiry is found to disclose elements of regularity, 
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suggesting a law or laws. This is the stage of inductive 
synthesis^ These laws are expressed, if possible, in the 
form of logical or numerical axioms, resembling those 
of Euclidean geometry. The methods of logic and 
mathematics are then brought into play to develop the 
consequences of the axioms, producing an assemblage of 
theorems or propositions. This department of the science, 
namely the posing of axioms and the deduction of theorems, 
is usually called the pure branch of the science. Even if 
future observations should invalidate the axioms extrinsi- 
cally, the discrepancies between theory and fact being 
too great to be explained away, these axioms and the 
deductions based on them would still have an abstract 
validity, as a logical structure of propositions exempt 
from self-contradiction; but for the description and 
explanation of the phenomena a new set of axioms would 
have to be found. On the other side, the corroborative 
part of the science consists in interpreting the abstract 
functions, formulae, equations, constants, invariants and 
the like, which occur in the pure formulation, as measures 
and measurable relations of actual phenomena, or numbers 
constructed from those measures in a definite way. This 
interpretative discipline constitutes the applied branch of 
the science. 

Such a division or dichotomy into pure and api)lied can 
be recognized in almost any science. A good example is 
Newtonian dynamics, according to which the motions of all 
bodies in the universe were presumed to obey certain axioms 
and postulates, namely Newton’s laws of force and motion 
and the law of gravitation. Later experiments, more^ 
numerous, more delicate, more comprehensive, suggested 
that this formulation, though describing almost all observed 
dynamical phenomena with a precision unprecedented in 
history, did not sufficiently account for certain exceptional 
facts, such as the precession of the perihelion of Mercury. 
The discrepancies between prediction and actuality were 
extraordinarily small, but they were persistent. There thus 
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arose a theory, or rather a succession of supplementary theories, 
of relativity, formulated on a new axiomatic basis by which 
the discrepancies of the earlier one might be reconciled, or 
removed. This reformulation of hypotheses still proceeds, is 
still incomplete, and undergoes modification from time to time. 

What is the axiomatic basis of the science of statistics, 
and what are the facts upon which the inductive synthesis 
is based ? The facts are certain regularities which have 
been observed in the proportionate frequency with which 
certain simple events happen or do not happen, when the 
circumstances under which they may occur are reconstructed 
again and again in repeated trials ; and the axioms, and 
the structure of theorems founded upon them, constitute 
the subject called mathematical probability. As for the 
facts, anyone who is interested can collect a few for him¬ 
self. Spin an ordinary coin a large number of times, 
and one can hardly fail to notice that the proportions 
of heads and of tails are very nearly equal; or shako 
a well-made die repeatedly from a dice-box and one will 
find that after many trials each face of the die has turned 
up in about one-sixth of the total number of trials. 

Example. The reader is recommended to experiment 
with simple repeated trials of this kind, and for future 
reference to record the results in sequence, in the order in 
which they occur. For example, the record of spins of a 
coin might be 

00101 OHIO 01101 00001 10111 ... 
or the like, where “ 1 ” denotes “ heads,** and “ 0 ** “tails.” 

It is instinctive to look for some cause for this 
^approximate equality of frequency in heads and tails, 
and natural to locate this cause as somehow resident in 
the two-sided nature and appreciable symmetry of the 
coin ; or to ascribe the approximate equality of frequency 
of the faces of the die to its six-sided and nearly uniform 
configuration. Simple ideas such as these suggest by 
generalization and abstraction the axioms of probability; 
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but the choice of axioms may be made in various ways, 
which lead to different formulations of the theory of 
probability. 

3. Survey of Various Definitions of Probability. 

No single particular definition of probability has so far 
met with predominating acceptance. The requisites of a 
satisfactory basis would be these : breadth of application, 
sufficient closeness to the intuitions in which the concept 
originates, and freedom from excessive complexity or 
abstruseness. No theory as yet proposed has been able 
to make these requisites compatible. We may survey 
some contrasting standpoints. 

Probability as the Logic of Uncertain Inference. 
One view is that probability may be regarded as a kind 
of extension of classical logic, an extension conveniently 
described as the “ logic of uncertain inference.’’ This 
view has been expounded by J. M. Ke 3 mes in A Treatise 
on Probability (London, 1921), especially in Part II, 
Chapters X-XVII, where references to earlier expositions 
are given. Probability is here regarded as “ the degree 
of our rational belief ” in the truth of a given proposition, 
such belief being contingent on a body of relevant know¬ 
ledge. A logical algebra is developed, but the theorems 
are stated in symbolic, not in numerical or metrical terms, 
and can be applied to the objective problems of statistics 
only by an abrupt and dubious transition from the symbolic 
to the metrical. 

Probability k Priori, and Probability as Relative 
Frequency. As our simple illustrations of the coin and 
the die have suggested, the crude intuition of probability^ 
rests on the observation that when a given set of circum¬ 
stances S, such as a symmetrical coin spun rapidly, has 
been present on numerous occasions in the past, it has 
been associated in a nearly constant proportion of those 
occasions with some event E, such as the fall of “ heads.” 

The apriorist theory directs attention to the set of 
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circumstances S, or rather to the invariant part of 8, In 
many spins of a coin or die something remains unchanged, 
namely those properties which describe the coin or die 
as a rigid constant configuration. The apriorist will 
regard the probabilities of falls 1, 2, 3, 4, 5, 6 of a die 
as some part of the description of the die, as measuring 
indeed some quality resident in the structure of the die, 
before any spinning is performed. Now the classical 
a priori definition took account only of a very limited 
class of “ systems ” 8, namely those possessing symmetry, 
in the sense that the different aspects (such as faces 1, 2, 
3, 4, 5, 6 of the die) were presumed physically indistin¬ 
guishable. Such an assumption is an idealization of the 
facts, for we can never hope to test completely the 
symmetry of any actual coin or die ; not only would 
the tests be infinitely many and impossibly delicate, but 
the concept of the rigidity and permanence in time of a 
material body is not sustained by modern physics. How¬ 
ever, symmetry being presumed, the six faces 1, 2, 3, 4, 
6, 6 were characterized as “ equally likely to be found 
uppermost after any throw, and the probability of 1/6 
was attributed to each of these “ events.'' More generally, 
if n equally likely aspects of a proposed system 8 were 
discriminated, m of these being favourable to the event E, 
the probability of E with respect to 8 was defined as 
p(E ; 8) = mjn. 

Criticism is easy. The logician will not fail to pounce 
upon the words “ equally likely,” pointing out that they are 
synonymous with “ equally probable,” and that therefore 
probability is being defined by what is probable, a circulus 
kin definiendo being thus committed. Postponing the 
defence, we may pass on to inquire what could be the 
definition of probability, should the tests have disclosed 
asymmetry in 8, The inquiry is most pertinent, for the 
heterogeneous and the asymmetrical are the prevalent 
order of nature, the homogeneous and the symmetrical 
being the exception. One has no difficulty for example in 
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conceiving a die which might be an irregular hexahedron, 
heterogeneous in density and with non-parallel and unequal 
opposite edges and faces. Such dice, and more complicated 
asymmetrical systems, have been subjected to repeated 
trials, which have shown a tendency of relative frequency 
of falls towards a constancy resembling that observed in 
symmetrical systems. 

Stability of Relative Frequency. Another view 
from the angle of “ common sense,’* in some respects 
antithetical to the view just mentioned, is the frequency 
view. Here the invariability of the configurative part 
of S, whether symmetrical or unsymmetrical, is tacitly 
assumed, and attention is concentrated upon the sequence 
of trials, and the incidence of E in these. For example, 
the die is thrown again and again. When E occurs, let 
us write 1 ; when E does not occur, let us write 0. A 
succession of n trials then gives a sequence 

A == Q/-^ 0/2 U 3 ... 0/^f^y • • • (1) 

each Of being 1 or 0 . 

Let m be the number of I’s in this sequence. A very 
limited experience, such as spinning a coin or die 10 times 
on several occasions, will show that in a finite number n 
of trials made upon the same system S on two or more 
occasions, different values of m are not only possible but 
usual. Thus, if E is the throw of an ace with a single 
die, 100 throws may on one occasion give m = 15 and 
on another occasion give m = 20. It follows that in order 
to define a probability p(E ; S) which shall be unique and 
not discordant with experience, we must idealize once 
again, postulating a limiting process as n tends to infinity 
and writing 

lim m/n = p(E \ 8), , . . (2) 

n->oo 

This is in fact a definition, supported by a certain school 
of statisticians, based upon the limit of frequency ratio 
or relative frequency m/n. Though at first sight attractive, 
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it fades a little on scrutiny. Granted the postulate of this 
limit p for one sequence of trials upon S, can we accept 
the more stringent postulate that the same limiting value 
p is obtained for any other infinite sequence of trials on 
S ? Not without further assumptions, for one might 
imagine a mechanism sufficiently delicate to throw heads 
with a coin, or an ace with a die, on almost all occasions. 
There is therefore some restriction on the manner of 
throwing, or on the initial state of S. This restriction 
is usually stated in the form of a condition that successive 
throws must be “ random,” but this merely transfers the 
burden of explanation to a new and undefined concept, 
“randomness.” To discuss various attempts to define 
randomness would take us too far afield. It is easy to 
say that randomness is absence of any law ; but what is 
“ law ” in this connexion ? 

Another difficulty is that the tendency of relative 
frequency m/n towards a limit p is different in nature 
from the corresponding tendency to a limit which mathe¬ 
maticians have discerned and used in the infinite sequences 
of mathematical analysis. To take a classical example, 
in the sequence defining a certain simple geometric series, 

Ij 1~2> .(3) 

the deviations of the successive terms from f are respectively 
J, — J, ..., each being numerically half its pre¬ 

decessor, so that, given a small number e, such as 1/1000000, 
we can always find some term sufficiently far along the 
sequence, after and including which all terms deviate from 
f by less than €. Thus f is the limit of this sequence. 
* JBut what can be asserted concerning the sign and magnitude 
of the deviation considered as a function of n, in 

€n = mln—p(E : S) ? 

It would seem that the only kind of assertion about €„ 
which would carry conviction would itself involve some- 



PROBABILITY AND AGGREGATES 


9 


where the notion of probability ; and here the risk of 
committing a circle in definition again raises its head. 

It should be added that the chief defects of the approach 
to probability by limit of frequency ratio have lately been 
removed by the work of de Mises, Copeland, Dorge, Wald 
and others. These writers admit only certain sequences 
A of suitable postulated properties, including that of 
limiting ratio ; but some logical difficulties remain, and 
the modified formulations lose the primitive simplicity in 
which they originated. 

It would seem, however, that a more natural course, 
and one more in line with the general method of science, 
would be to try to explain the effecty namely the relative 
frequency of E, by an analysis of the causCy namely the 
system 8. This suggests a return to the a priori stand¬ 
point ; and it may be noted that several authors at the 
present time, Fr^chet, Kolmogoroff, Cramer and others, 
have been independently engaged in rehabilitating the 
a priori definition by furnishing it with a better axiomatic 
basis. 

4. Probability as Measure of a Sub-Aggregate. 

Let us examine more closely the system 8y keeping some 
simple system such as a coin or die in mind. The 
approximately constant element in our sequences A, 
namely the almost stable frequency ratio of E, must 
reflect—at least so our intuition suggests—the constant 
element of 8y such as the rigid configuration of a coin 
or die; the irregularity which we name randomness 
doubtless reflects the variable part of 8y such as the 
initial position, velocity and angular velocity of projection. , 

What is S when an unsymmetrical and heterogeneous 
die is spun and falls ? It consists of (i) the die, specified 
as a particular constant rigid body, (ii) the floor or table 
on which it may impinge or finally rest, (iii) the surrounding 
air, and so on ; together with (iv) the circumstances of 
projection, described by coordinates of initial position, 
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momentum and angular momentum. The coordinates 
specifying the rigidity of the die and the configuration 
of the table or floor are constant components of S, the 
other initial coordinates of 8 are variable. The set of 
coordinates of 8 at the instant of projection may be 
called the initial phase. Each variable coordinate, such 
as the initial position, or the initial momentum, has a 
certain field of variation. Hence we must assume a set 
of possible phases which, if they can be enumerated in 
some order, may be designated by 8-^, 8^, •••, ; 

and this ensemble of possible initial phases 8^ constitutes 
an aggregate 8 of the kind specially studied in pure 
mathematics.* If d 3 niamical determinism be assumed, 
but not otherwise, the initial phase will decide whether 
or not the event E will occur. Consequently the possible 
initial phases may be classified asi^-phases or not -J^-phascs 
(let us say J^-phases), so that the whole phase aggregate is 
divided into two sub-aggregates. Now the question of 
assigning a measure to such aggregates has been deeply 
studied in modern pure mathematics, the guiding idea 
being that of extending as widely as possible the scope 
of a concept familiar in simple cases, namely the cardinal 
number of a finite set of objects, the length of a line, the 
area of a surface, the volume of a solid. If M is the 
measure of the whole aggregate 8 of possible phases, and 
pM the measure of the aggregate of ^-phases contained 
in it, then p is the probability p(E ; 8), 

Something has been glossed over here ; there is the 
tacit assumption that the initial phases are equally 
likely.’’ But let us insist that the question of equal 
likeliness is not one for the abstract formulation at all; 
for to specify the aggregate is in effect to say that its 
elements, the initial phases, are equally likely. For 
example, if the aggregate were of points on a continuous 
line segment, and the measure were ordinary length, then 

* We use the same letter S as before, regarding the system 
now as the totality of its possible phases. 
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we have implied in this description that all points in the 
segment are equally likely. On the other hand, the question 
of equal likeliness is crucial in the application to experiment 
or observation, that is, in applied statistics, where a 
wrong choice of the aggregate may alter all the pro¬ 
babilities. This has long been known in problems of 
so-called geometrical probability. For example, given a 
circle, let a chord be drawn across it at random : what 
is the probability that the length of the chord exceeds 
half the diameter ? It depends entirely on the manner 
in which the chord is drawn. If it is done by taking a 
point on the circumference and then drawing the chord 
at any angle, all angles being thus supposed equally likely, 
then the probability is 2/3 ; but if it is done by taking 
any diameter and drawing the chord at right angles to 
any point taken in the diameter, the diameters and points 
being equally likely, then the probability is \/3/2. 

The inclusion of the words “ equally likely in a definition 
is in fact a concession; it puts the reader more gently at 
terms with the abstract formulation by anticipating its chief 
future application. The usage is not uncommon. When a 
point is defined as “ that which has position but no magnitude ** 
the same appeal is made to an application, but the same 
suspicion of a circle in definition is incurred, for how can 
position bo defined without the notion of a point ? And if a 
straight line is defined as “ lying evenly between its extreme 
points, what else does “ evenly moan but “ in a straight 
line ” ? Every definition which is not pure abstraction must 
appeal somewhere to intuition or experience by using some 
such verbal counter as “ point,” “ straight line ” or “ equally 
likely,” under the stigma of seeming to commit a circle in 
definition. 

This prologue, though it has omitted many subtler 
points which could be amplified at very great length, 
must now be cut short. To summarize : (i) events E 

are conceived as associated with, or caused by, phases 
Si of circumstances ; (ii) each Si gives rise unambiguously 
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either to E or to E ; (iii) the phases 8i form in their 
totality a set or aggregate 8, of which the phases favourable 
to j&, and those favourable to E, form complementary 
subsets; (iv) a measure M can be given to the whole set 
8, and if pM is the measure of the subset favourable to Ey 
then p is the probability p{E ; 8) of E with respect to 8 ; 
(v) the question of equal likeliness of phases is the same 
as the question of specifying the aggregate and its measure, 
and in practical applications this must be determined by 
the circumstances of the particular problem. Let us 
finally add that the word phase can be extended to include 
coordinates other than dynamical ones; also that the 
name “ fundamental probability set ** is used by some 
writers for the set 8 of phases 8i, 

6. Definition of Probability. In an elementary 
treatment a rigorous formulation in terms of general 
aggregates is not possible. It will be necessary to restrict 
consideration to aggregates with a finite number of elements 
only ; in this case the measure of an aggregate or sub¬ 
aggregate is simply the number of elements it contains. 
The reader may take it that the theorems can be extended 
to more general aggregates. 

Definition. If an event E can result from the phases 
of a system 8, there being n different phases and no more, 
all equally likely d priori ; and if m of those phases entail 
the occurrence of E (so that n—m do not), then min is the 
probability p{E \8) oiE with respect to 8. 

Continuous Case. If the event E is described by 
the value of a continuous variable x, we may denote the 
probability that x is found between x-\-\Ax and x—^Jx by 

p(x-i-iJxy x—iJx; 8) = Ap(x \8)y . . (1) 

let us say. By supposing n to tend to infinity and Ax 
to tend to zero we reach the conception of a ^flferential 
element of probability, or probability differential, 

p(x-\-\dXy x—\dx ; 8) = dp{x ; /S), . . (2) 
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which, when no misunderstanding about S is likely to 
arise, we shall often denote briefly by dp. 

Complementary Event. The failure of E is denoted 
by E, and is called the complementary event. The pro¬ 
bability of E is (n—m)ln, namely 1—p in the finite case, 
and likewise in the continuous case. This is often termed 
the complementary probability and denoted by g, so that 
p+q == 1* 

If n is finite and if E must inevitably happen in all 
of the n ways, then p — \ and is “ certain,** while 
g = 0 and 2 is “ impossible.** If, however, the system 
8 depends on a non-finite set or results in events expressible 
by a continuous variable, we must not suppose that p = 1 
implies certainty, or p = 0 impossibility. For example, if 
a point is taken on a line segment, the chance of a particular 
point P being taken is 0 ; but some point is taken, and so 
the point P cannot be regarded as impossible. 

6. Addition and Multiplication of Probabilities. 
Dependent and Independent Events. An event F will 
be said to be dependent on an event E when the happening 
of either E ov E alters the probability of F ; and in the 
contrary case F will be said to be independent of E. An 
extreme case of dependence is that in which the happening 
of either E ov F makes the probability of the other equal 
to zero. The events are then said to be mutmlly exclusive. 
(In the continuous case we must take cognizance of “ almost 
mutually exclusive ** and “ almost independent *’ events, just 
as we have of “almost impossible** events for which p == 0.) 

The addition theorem of probability is applicable to 
events which are mutually or almost mutually exclusive. 

Theorem. When an event E may happen in the 
form of any one of r mutually exclusive events 
= 1, 2, 3, ... r, in a system 8 which has n equally likely 
phases, the probability of E^ being p^, then the probability 
of P is 

p(E \ 8) = pT^-\-p2+-.-+Pr == . . (1) 
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Proof. If of the n phases entail then Pf = nfjn. 
Since the phases do not overlap (otherwise the events E^ 
would not be mutually exclusive) the total number of 
phases entailing one or other of the E^ is ; and so 

i 

p(E ; S) = Znjn = Ep^, 

i 5 

The theorem, which is sometimes called the theorem of 
Total Probabilityt continues to hold for systems expressed 
by a non-finite n or by a continuous variable. 

The multiplication theorem, or theorem of Compound 
Probability, refers in the first instance to independent 
events, but can easily be made applicable, with a suitable 
definition of conditioned probability for dependent events, 
to the latter case. 

Theorem. If -£7^, j = 1, 2, 3, ..., r, are r independent 
events, each with respect to its own system Sf, the 
probability that they all happen when all the 8^ are in 
operation is 

p{E-,8)=pjp,...pf, . . , (2) 

where E denotes the compound event consisting in the 
happening of all the E^, 8 denotes the compound system 
consisting in the operation of all the 8^, andp^ = p(Ef ; 8^), 

Proof. Let denote the number of phases of 8^, 
and of these let entail E^, Now each of the phases 
of 8j may be paired in turn with each of the phases of 
8]ci giving rise to compound phases of the double 
system (/S^, /S'*). By similar reasoning the phases 
entailing E^ may be paired in turn with the mj^ phases 
entailing Ej^, giving rise to compound phases of 

(8f, 8 t^ entailing the double event (E^, Ej^. 

By similar reasoning, or step by step, there are 
altogether phases of the compound system 

(/Si, /Sg, ..., 8^) = 8, and of these entail the 

compound event (E^, E^, E^) ==E, Hence the pro¬ 

bability of E with respect to 8 is 
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'p{E ; 8) = 

= PiP2--Pt^ 

Once again we must content ourselves with the state¬ 
ment that the theorem remains true for independent or 
“ almost independent ” systems involving infinite aggregates 
or continuous variables. 

By modifying the definition of p^y p^y Pr we 
may prove an analogous theorem for a chain of events 
Ely E^y ...| E^y each of wliich infiuences the probability 
of its successors. 

Let P 2 = p(E 2 ; Eiy 8^ denote the probability of E^ 
after Ei has happened, p^ = P(EI ^; Eiy E^y 8^) denote the 
probability of E^ after Ei and E^ have happened, and so 
on. Slight consideration will show that this simply 
involves putting the events in an order of time and that 
then, with the new interpretation of p^y ^ 3 , ..., pj., the 
above proof proceeds exactly as before. Hence we have 
the theorem of compound probability for a chain of 
conditioned events: 

p(E ; 8) 

=p{Ei)p(E^; Ei)p(Ea; E^, E^) ...piE^; Ei,E^, ...,E^^). (3) 

These theorems of addition and multiplication of 
probabilities are the fundamentals upon which the mathe¬ 
matical theory of statistics is raised. Since addition and 
multiplication are operations of ordinary algebra, we may 
anticipate that there is an algebra of probability depend¬ 
ing on these operations, according to which expressions 
representing independent systems 8^ can be compounded 
in product and the resulting probabilities foimd by 
inspection of terms. This algebra is the algebra of 
generating functions of probability, which wo shall consider 
from an elementary standpoint in the next section. 

Ex. 1. The probability of throwing two consecutive aces 
with a true die is J 
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Ex. 2. The probability of throwing a head and a tail with 
two coins is 

Ex. 3. The probability of throwing a total of 8 points 
with two dice is 6/36. (The mutually exclusive events are 
6-f2, 5-f"3, 4-1-4, 3*f6, 2-f-6.) 

Ex. 4. A bag contains 4 black and 3 white balls. Show 
that the probability of drawing 3 black in succession is 64/343 
if the ball drawn is replaced each time, 8/49 if the first ball 
drawn is replaced but not the others, 4/35 if no ball is replaced. 

Ex. 6. The events and Eg are neither independent nor 
mutually exclusive. Denote by Pig probability that 
and Eg both happen. Prove that the probability that at 
least one of Ej and Eg happen is Pi-hP 2 —‘Pi 2 * 

Ex. 6. Generalize the preceding theorem to r events 
El, Eg, ..., Ef, Provo, with an analogous notation, that the 
probability that at least one of the events happens is 

Upj 22Jpjii-\-ZZUpjj^ ... ( 

7. Generating Functions of Probability. We shall 
often denote the probability that a variable x takes a 
particular value by and we shall use the following 
nomenclature : 

Probability Function. The function is the 

probability function. When the set of values of z is 
continuous we shall write the probability differential 
dp = <l){z)dz for the probability that z is found in the 
range {z—^dz, x-\r\dz). In this case <l>[z) is often called 
the probability density. 

Variate. A variable which has a probability function 
will be called a variate. 

Generating Function. Associated with <j){z) we 
introduce the generating function (g.f.) of probability, 
defined by 

G(t) = G{t; <f>) = ^<l>{Xj) fi . . • (1) 

for variables which take discrete values, and by 

0 (t) = ((l)(z)t^dz . . . • (2) 
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for continuous variables, the integral being over the whole 
range of possible values of x. 

Ex. 1. The generating function of probability for heads 
in a symmetrioal coin is 

Ex. 2. The g.f. for a symmetrical six-sided die is 

Ex. 3. The g.f. for an unsymmetrical coin in which the 
probability of heads is p, of tails q, is pt+q. 

Ex. 4. Write down the g.f. for a symmetrical four-sided 
die ; also for an unsymmetrical one in which the probabilities 
of faces marked 1, 2, 3, 4 are Pi, Pi^ Vi' 

Ex. 6. If all points on the straight line from a: = 0 to 
X = 1 are equally probable, the g.f. is 

J* t^dx = (i--l)/log^^ 

8 . Properties of Generating Functions. Suppose 
first that we have an event and its complement 
of respective probabilities p^ and q^, and a second indepen¬ 
dent event E<^ with its complement E^, of probabilities 
P2 9^2- Then the compound probabilities of the four 
mutually exclusive events 

(^ 1 , E^), {E^, E,), {E,, (E,, E,) . . (1) 

are respectively PiP2> Pi?2» ^iP2y M2* relate these 

to the terms on the right of the algebraic identity 

{Plh~^^l)(P2h~^22) == PlP^^^2+Pl^2h+^lP2^2~^^1^2' ( 2 ) ^ 

Study of this identity will reveal the most important 
property of generating functions. The disjunction between 
added terms (those linked by plus signs), both in the factors 
on the left and in the expanded product on the right, 
reflects in each case the disjunction into a number of 

B 
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mutvxilly exclusive events. The operations of multiplication, 
on the other hand, are carried out on expressions symbo¬ 
lizing independent events. For example, the multiplication 
of the two factors on the left interprets the compounding 
of the two independent systems 8 ^ and iSg of which they 
are the generating functions ; and the results of multi¬ 
plication visible in single terms on the right, such as 
PiPj^t^i represent at the same time the compounded 
probabilities, PiP2i and the compounded events, 
characterizing (E-^, In fact the algebraic operations 

are faithfully carrying out the consequences of the two 
basic theorems of probability. Mere inspection will 
convince us that this is true not only for binomial 
expressions compounded in product as above, but for 
multinomial expressions, as in the following example. 

Ex. 1 . Let the reader consider events E^, E^, E^ of pro¬ 
babilities p^, Pg, Pg with respect to 8 , events E*^, E'^, E'^, 
with probabilities p^, Pg, p^, p^ with respect to an independent 
system 6", and examine the product 

+^ 2*2 

in relation to the 12 events of the compound system 

T= (8, 8'). 

Regarding a compound system { 8 , 8 *) as a single 
system and introducing further independent systems one 
at a time, we may prove step by step that to find the 
respective probabihties of all the mutually exclusive events 
arising from the compounding of r independent systems, 
^ we must construct the product of r expressions of the 
kind exemplified above, and examine the individual terms 
of the expansion. 

Ex. 2. In an expansion of three factors such a term as 
interpreted as meaning that the com¬ 
pound event {E^, E'^, E*^) has probability P^P^P^^ 
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The variables and so on are introduced for the sole 
purpose of preventing the terms from being merged 
together ; for when the are explicit fractions such as 
I and the like some such device is needed. 

Now suppose the event involves the addition of Xj 
points to a score, or the assumption by an additive variate 
X of an increment x^. In such a case we represent Ej by 
rather than by t^, taking advantage of the fact that when 
expressions like pjt ^5 and are multiplied together we 
have by the law of indices the probabilities 

being multiplied as they ought to be, and the increments 
Xj and Xjc being added as they ought to be. With this 
understanding, the system under which x may assume 
values Xf with probabilities j = 1, 2, r, is char¬ 
acterized by the expression 

2:pit^i .(3) 

i 

But this is merely the generating function 0 (t) of the 
system, and so we infer the important theorem, for 
discrete variates in finite sets : 

The g.f. of a compound of independent systems is the 
product of the gj's of the separate systems. 

By a limiting process, with due precautions on the 
functions concerned, this multiplicative law can be extended 
to g.f.’s involving continuous variables. Thus, if G^{t) is 
the g.f. of the variable x, and GgCO of a statistically 
independent variable y, then Oi(t) 0 ^{t) is the g.f. of x+y ; 
and so for more than two variables. 

Ex. 3. The probabilities of 3 heads, 2 heads, 1 head and 
no heads in a throw of three symmetrical coins (or three 
separate throws of one coin) are the coefficients of t and 1 
in the expansion of namely f, f, J respectively. 

Verify this also by enumeration of cases. (Write H for head, 
T for tail; then the cases are HHH; HHT, HTH, THH ; 
HTT, THT, TTH ; TTT.) 

Ex. 4. The corresponding probabilities when the coin is 
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unsymmetrical, with probability p for heads and q for tails, 
are the coefficients in the expansion of (pt-\-q)^. 

Ex. 6. The probabilities when the three coins are different 
and unsymmetrical are the coefficients in the expansion of 
(Pi^+?i)(P2<+g2)(P3<+?a)* 

Ex. 6. The probabilities of n, n—l, 2, 1, 0 heads in 
n throws of an unsymmetrical coin are the coefficients of 
powers of t in the expansion of (p^+g)**. 

Ex. 7. Write down the corresponding g.f. for the 
simultaneous throw of n different unsymmetrical coins. 

Ex. 8. A tetrahedral, a cubical and an octahedral die, all 
symmetrical, are thrown together, their faces being numbered 
in each case from 1 upwards. Show that the probabilities of 
totals 3, 4, 18 are arrayed by coefficients in the expansion 

of 

i-i-id 

Ex. 9. A coin is thrown n times. Each time a head 
occurs, 2 is added to the score; each time a tail occurs, 1 is 
subtracted. The g.f. is 

Ex. 10. Four tickets marked 00, 01, 10, 11 respectively 
are placed in a bag, and drawn one at a time, being replaced 
each time. Prove that the chance of drawing five times and 
obtaining ticket numbers summing to 23 is the coefficient of 
in the expansion of = 4-®(l+^)®(l+w)*. 

Find this coefficient, and verify the result by enumeration. 

9. Moments and Moment Generating Functions. 

It is convenient to describe a probability function ^[x) 
by certain coefficients or parameters connected with it, 
such as moments, cumulants and others later to be defined. 
The moments commonly employed are based on powers of x, 
and are defined by 

fjLr == 2 x^(f){x) or jx^(l>{x)dx, . . (1) 

according as the variate is discrete or continuous. The 
summation or integration is over the whole range of 
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possible values of x. If the values which x can take are 
discrete and spaced at unit intervals (for example if x 
records the number of heads in n throws of a coin) it is 
mathematically preferable to use factorial momenta, defined 

by 

M(r) ~ 

where x^^^ ~ x{x—l)(x—2){x—r-\-\), , (2) 

Note. The privilege often accorded to ordinary “ power ” 
moments is one of custom only ; no special sanctity attaches 
to them. 

Mathematical Expectation. If f(x) is a function of 
X, and ^(x) is the probability function, or (f>{x)dx the 
probability differential, then the sum or integral 

Zf{x)(f>{x) or J f(x)<f}{x)dx . . . (3) 

is called the mathematical expectation of f(x). It is often 
denoted by Ef{x). The moment is therefore the 
mathematical expectation of x'^. 

Moment Generating Functions. If we put t = e°’ 
in the g.f. of probability 0(t), we obtain 

G(e®) == 2(f)(x)e^^ or J ^{x)e^Hx . . • (4) 

= l+i^ia-l~ja2a“/2!+)a3a7:il h-.-, 

provided that the sum or integral converges over a range 
of a and that expansion of e^^ and integration term by 
term is permissible. This function, which we shall denote 
by iff (a), may be regarded as generating the moments p,', 
in the sense that is the coefficient of a'^jrl in iff (a). 
Of course a, hke t, is a variable introduced to facilitate 
manipulation, in fact to carry the moments. We shall 
call iff(a) the moment generating function (m.g.f.) of x or 
of <}>{x), or of the system under consideration. 
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Factorial Moment Generating^ Functions. When 
factorial moments are in question, we can construct a 
factorial moment generating function (f.m.g.f.) very simply 
from the probability g.f. by the substitution i = 1 +a. 
For then we have 

0{l+a) ^ Z<i>{x){l+aY .(5) 

by expanding (1 +a)* by the binomial theorem and summing 
the resulting terms. 


Example. The f.m.g.f. of the distribution characterized by 
(pt-\-q)^ is (1+pa)". 


Note, The reader who is acquainted with more advanced 
mathematics may observe that for moment generating 
functions the substitution t = instead of < = e® has a 
certain advantage. It gives the modified m.g.f. 




. (6) 


a Fourier transform of ^{x). The integrand and integral are 
bounded, and the reciprocal theorems of Fourier transforms 
are available. 

10. Gumulants and Cumulant Generating Func¬ 
tions. If the logarithm of the moment generating 
function M(a) can be expanded as a convergent series in 
powers of a in the form 

K(a) = log^if(a) = /Cia+#C2aV2!+^3a®/3!+ ..., . (7) 

then K{a) is defined to be the cumulant g.f,, and the 
coefficients are called the cumulants * of the function 
Since m.g.f.’s are compounded in product, c.g.f.’s 
must be compounded in sum, whence the theorem; 

When independent systems are compounded the r^^ cumu- 
lants Kr of the separate systems are added to form the r^^ 
cumulant of the compound system. 

This additive property of cumulants is indeed the 


♦ The word “ cumulant,” suggested by R. A. Fisher, is to bo 
preferred to the older term “ seminvariant, since “ seminvariant ” 
is already appropriated in the theory of algebraic invariants. 
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reason for introducing them. In the same way, by taking 
the logarithm of the f.m.g.f. we can define a factorial 
c.g.f. and factorial cumulants. 

Example. The factorial cumulants corresponding to 
are np, —np^, 2lnp^, —3lnp^ and so on. 

11. Change of Origin and Scale in Generating 
Functions. Change of Origin. If the origin from which 
the variate x is measured is transferred from a; = 0 to 
X = a, any value x will be changed to x—a. Hence 
every factor in a term of the probability g.f. will become 
tx-a . the accompanying probability though 

changed in notation, will not be changed in value. Hence 
the effect is to multiply the whole g.f. by 

This very simple rule leads to corresponding ones for 
the m.g.f., f.m.g.f. and c.g.f., namely : 

A change of origin from x = 0 x = a has the effect 
of multiplying the m,gf, by e"^® ; of multiplying the f-m.g.f. 
by (1+0.)“^ / ctnd of adding to the c.g.f. the term —aa. 

Thus only the first cumulant icj is changed ; it becomes 
K^—a, while Kg, Kg, ... are unaltered. 

Change of Scale. If the scale of measurement is 
altered so that what was previously recorded as x now 
reads kx, then every factor t^ in the previous g.f. now 
becomes that is, (t^)^. Hence in the m.g.f. the previous 
e®* now reads e®*®. Hence the rules : 

Change of scales so that x becomes kx, has the effect of 
replacing t by t* in the probability g.f., a by ka in the m.g.f. 

The immediate consequence is that the previous 
moment /a' and r*^ cumulant k^ become k'^p!^ and k^K^. 

The reason for the older name seminvariant is now seen ; 
for imder a change of origin and scale in x all cumulants after 
Kj are altered at most by a scale factor. 

Change of scale in the f.m.g.f. will be effected by 
replacing 1+a by (1+a)*. 


Example. The first moment or mean of the distribution 
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which has g.f. is np, and the m.g.f. with respect to 

the mean as origin is The corresponding 

f.m.g.f. is (1 + (1 +pa)”. 

12. Population, Universal, Universe or Stock, 
Sample. To conclude these questions of nomenclature 
and general notions we explain what is meant by popula¬ 
tion, universe or stock, and sample. As an example let 
us consider the repetition of an experiment in which the 
probability of success is p = mjn, a rational f5paction. We 
may construct a model by taking (or imagining) n similar 
objects, such as equal spherical marbles, of which m are 
distinguishable from the rest, and drawing an object 
repeatedly, with replacement after each drawing. Such 
an assemblage, actual or hypothetical, constitutes a 
population, universe or stock. It is in fact merely a model 
of the system 8. To cope with special cases we have 
often to conceive a fictitious infinite population. For 
example, if we wish to represent drawing with replacement 
by a model in which the drawing is without replacement, 
the population of the model will certainly have to be 
infinite, since the probabilities of successive drawings are 
constant, a thing which cannot happen with a finite 
population. 

Sample. Any element of a population is a sample 
of that population. For example, if five drawings are 
made, with replacement each time, from six cards numbered 
1, 2, 3, 4, 5, 6, the population of possible sets of five cards 
contains 6® or 7776 elements, of which (3, 6, 5, 4, 1) and 
(4, 4, 2, 6, 3) are two samples. If the drawing is without 
replacement, the population of sets of five contains 
6.5.4.3.2 or 720 elements, of which (2, 3, 5, 6, 4) and 
(6, 2, 4, 3, 1) are two samples. Or again, if a coin is spun 
100 times, the sequence of heads and tails arising is to be 
regarded as one sample out of the possible 2^®® sequences 
constituting the population of sequences. 

The word “ sample ” is also used as a verb, “ to sample ” 
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a population meaning to draw a sample, or samples, from 
that population. 

Notation. It is important to distinguish the pro¬ 
bability <f>{x)y which may not be definitely known, from 
the relative frequency of x as found in a sample, let us 
say f{x ); and in the same way all parameters, such as 
means and moments, connected with <f>{x) should be 
distinguished from the corresponding parameters in the 
case of/(a:). As far as possible we shall make this distinc¬ 
tion by using Greek letters for probability functions and 
parameters, italic letters for the corresponding frequency 
functions and parameters. Thus if /x' stands for the 
moment of <f)(x), then will be the moment of f(x) ; 
and so on. 

For detailed description of many aspects of theoretical 
and practical statistics, and for bibliographical references to 
memoirs and texts on the subject, the reader may consult 
An Introduction to the Theory of Statistics, by G. U. Yule 
and M. G. Kendall, London, 1937, the 11th edition of the 
original book by the first-named author. 

For an account of moments, factorial moments and 
cumulants. Chapter 3 of M. G. Kendall’s The Advanced 
Theory of Statistics, London, 1943, may be consulted. 

In this book, to cover the topics within a limited 
space, we have made a systematic use of moment generating 
functions. In strictness a mathematical preamble would be 
required, setting out the conditions under which such integral 
transforms exist, and the conditions under which they may 
be uniquely reciprocated to the probability functions. The 
student intending to read advanced statistics will be well 
advised to gain as much preliminary knowledge as possible 
concerning Laplace and Fourier transforms and their inversion. 
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PROBABILITY AND FREQUENCY DISTRIBUTIONS : 
GRAPHICAL REPRESENTATION : CALCULATION 
OF MOMENTS 

13. Distributions, Probability Curve, Histogram. 
The assemblage of values of probabilities <l>(x), for all the 
possible values of x that may occur in any system 8, 
is called the probability distribution of x in 8. In practice 
a set of n observations in a sample does not usually give 
all the possible values x^, and certainly cannot give them 
all if they cover a continuous range. Further, the sample 
of n values is itself only one member of the population, 
often prodigiously large or even infinite, of possible samples 
of n values that might have been drawn. 

The relative frequency of x^ in a sample of n values is 
denoted by f(x^). The assemblage of relative frequencies 
f{Xi) for the sample is then called the frequency distribution 
of x in that sample. The name is also often given to the 
assemblage of absolute or actual frequencies, but these are 
merely obtained by multiplying all relative frequencies 
by n. 

Ex. 1. In repeated throws of a symmetrical coin the 
respective probabilities of runs of 1 head, 2 heads, 3 heads, 
... are ••• • Hence in 400 throws the ideal 

probability distribution may be tabulated (to the nearest 
integer) as : 

X 1 2 345678 Total 

50 25 13 6 3 2 1 0 100 

In an actual experiment of 400 throws (performed by the 
26 
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author) there were 196 heads, and the frequency distribution 
distributions of runs of x heads was : 

X 1 2 345678 Total 
n/ 61 24 14 4 6 1 0 1 100 

Comparing the actual with the theoretical distribution, 
the reader will note a fairly close agreement, and also a slight 
irregularity in the frequencies. 

If a; is a continuous variate, the curve y = is 
called the probability curve of x. (The term “ frequency 
curve will often be found, but it is not strictly accurate. 
Cf, 12.) The curve may be symmetrical about its central 
ordinate ; or it may have the “ long tail ” to the positive 
or right side, in which case it is said to be positively skew ; 
or to the negative or left side, in which case it is negatively 
skew. In some cases, as in the probabilities of runs of 
heads just considered, the curve may not descend at all 
on one side or the other. A curve so extremely skew is 
called positively J-shaped, or negatively J-shaped, as the 
case may be. In a rare type of distribution called the 
JJ-shaped curve the minimum ordinate is in the middle 
region. The area under a probability curve measures the 
total probability of all possible values of x, and is therefore 


equal to 1. 


yx. 

Neg. J-shaped. 

Ncg. skew. 

Symmetrical. 

Po8. skew. 

Pos. J-shaped. 

U-shaped. 


If a; is a discontinuous variate the plotted points {x, y), 
where y = (f){x), do not form a curve. The sum of the 
ordinates is equal to 1. It is customary, though there 
is no very cogent reason for doing so, to join these points 
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each to its neighbour by straight lines, thus obtaining the 
probability polygon for the distribution in question. The 
terms “symmetry” and “skewness” then have corre¬ 
sponding meanings. 

Frequency Polygon, Histogram. In an actual 
sample of observations we have relative frequencies instead 
of probabilities. If the variate x is discontinuous, as for 
example the number of flowers on stalks, the number of 
beans in bean-pods, we obtain separate plotted points 
{Xtf{x)) which, joined to their neighbours, form a frequency 
polygon. 


Frequency Polygon. Histogram. 

On the other hand, x may be a continuous variate, the 
range of which in the process of measurement is broken 
for convenience into intervals of flnite breadth. For 
example, height of men, measured in inches, is a continuous 
variate ; all heights within a certain range are conceivable. 
But in practice heights may be recorded to the nearest 
inch, in which case all individuals of the sample having 
heights in the range 66*5000... to 67*4999... inches form 
a frequency group or frequency class corresponding to 
X = 67, the central point of the class. In such a case 
it is customary to represent the class graphically not by 
a single ordinate at the central point but by a rectangle 
on the class-interval (as 66*5 to 67*5) as base and of area 
proportional to the class frequency or relative frequency 
f{x). The flgure of juxtaposed rectangles is then called 
the frequency histogram or simply the histogram (that is, 
diagram made up of cells), and it furnishes a rough 
approximation to the ideal probability curve. 

Ex. 2. Plot the probability polygon for the runs of heads 
in Ex. 1 ; also the frequency polygon of the experiment. 

Ex. 3. Note that often great care must be taken to 
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ascertain the exact class-boundaries and centres of classes. 
For example, the British Anthropometric Committee (Report, 
1883, p. 266) measured the height of 8585 adult males in the 
British Isles, made up of samples of 6194 from England, 
1304 from Scotland, 741 from Wales and 346 from Ireland. 
The distribution of the Irish sample reads as follows : 

X 69 60 61 62 63 64 66 66 67 68 69 70 71 72 73 n 

n/ 1 0 2 2 7 15 33 68 73 62 40 25 15 10 3 346 

When we are told, however, that the class a; — 59 inches 
means “ 59 and over,” but at the same time that measuremonts 
were to the nearest eighth of an inch, it appears that class 

a? = 69 means from x = 68}f to so that the centre of 

the class is at a; = 69^^ ; and so for every other class. 

The reader should draw the histogram for the above 
distribution, choosing not too small a scale for the 
frequency. 

For ease and rapidity in computation we can always 
by a change of origin take any convenient value of x as 
new origin, and by a change of scale make class intervals 
of unit breadth. At the end of any calculations we can 
translate the results back to the proper origin and scale. 
It is often convenient to choose a provisional origin either 
near the middle values of x or at one or other end of the 
range. 

Ex. 4. In the distribution of Ex. 3, if 67 is taken as new 
origin for x, the classes range from a; = — 8 to a: == +6. If 
these classes are presumed to be centred, the origin is not 
67 but 67^. 

14. Descriptive Parameters of Distribution. Pro¬ 
bability and frequency distributions may bo described, 
not completely, but in their main features, by the values 
of their moments, factorial moments or other parameters. 
Some of these parameters have a geometrical significance. 

Typical Parameters or Averages. There are three 
of these in common use, the mode, the median and the 
arithmetic mean. 
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Mode. The mode is the value of x for which the 
probability or in a frequency distribution the relative 
frequency fix), is a maximum, that is, greater than the 
probability (or frequency) on either side. In a probability 
curve it is the abscissa of a maximal ordinate. 

Many curves have a single maximum near the middle; 
others may show two maxima or more. These are called 
dimodal or multimodal, as the case may be. 

Mode. Dimodal Curve. Median. 

Median. The median is that value of x which divides 
the sum or integral of the probabilities over the whole 
range into two equal parts. This sum or integral must 
be equal to 1 ; and so if the range of values of x is from 
X ^ ato X = b, the median value of x is defined by 

Z<f,{x) ^ E<l>{x) ^ \hi>(x) . . (1) 

ax a 

rz rb 

or I <l>{x)dx — ^{y)dx = \ <f>(x)dx — . (2) 

J a J X J a 

For a continuous probability curve the median ordinate, 
by (2), bisects the area under the curve. 

Arithmetic Mean. The most widely used typical 
measure is the arithmetic mean, which is simply the 
Urst moment or mathematical expectation of x, namely 

fjL\ = I!x(f>{x) or jx(l>{x)dx. . . (3) 

These formulae are the same as those occurring in 
dynamics for the centroid of a series of particles of masses 
j>{Xj) placed at points x^ along a straight line, and the 
centroid of a straight rod of density <f>{x) at the point x, 
Xt follows that the arithmetic mean is the abscissa of the 
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ordinate through the centroid of the area under the curve 

y = 

The arithmetic mean of the values in a sample is 
correspondingly 2xf{x). 

Remark. In many probability curves of slight or moderate 
skewness the median lies between the mode and the arithmetic 
mean, nearly twice as far from the mode as from the mean. 

Moments about the Mean. The arithmetic mean is 
so fundamental in theory and in practice that it is 
customary, once it has been determined, to take it as a 
new origin and to refer all higher moments to this origin. 
Moments about thc^ mean as origin are usually denoted 
by undashed /x^. We find easily, by binomial expansion, 

fXf — U(x—iJi[y(l>(x) or j{x—iJL^y(f)(x)dx 

where denotes the familiar binomial coefficient 
r(r—l)...(r—5+l)/5 I. The last two terms can be merged 
into one as (—For example: 

/xi = 0, 

H'A = Mi-4/xi/x3+6(/xi) V2“3 (/x')*, . (5) 

formulae of regular application in practical work, since 
they hold equally well for moments of a frequency dis¬ 
tribution, <l>(x) being then replaced by f(x), and /x by m. 

Other means, such as the geometric and the harmonic 
mean, are very occasionally used with respect to rather 
special distributions. 

Gumulants in Terms of Moments about the 

Mean. By expanding the logarithm of 

if(a) =ef-‘'«(l+^2a2/2!+/X3a73!+... 
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as a series in powers of a, and comparing the coefiSicients 
of these with the coefficients in 10 (7), we find the rela¬ 
tions between cumulants and moments about the mean. 
The first four relations are 

Kl == K2 = ^3 = H'Sy ^4 = i^4 

15. Measures of Dispersion or Spread. Distribu¬ 
tions differ according as the values of x are spread densely 
or widely on either side of the mean. To describe this 
feature numerically we need parameters measuring 
dispersion. 

The arithmetic mean of the deviations from the 

mean is of course of no use for the purpose, being equal to 
zero. A measure occasionally used, but now falling into 
disuse, is the mean absolute deviation (the former name was 
“ mean error defined by the arithmetic mean of devia¬ 
tions from the mean all taken with positive sign, namely 

2\z-fi[\cl>(^x) or J|x—. . (1) 

where \x—fi\\ denotes the positive numerical, or absolute, 
value of 

Though usually computed with respect to it is 
actually in closer association with the median, in virtue 
of a certain minimal property, namely : 

The median value of x is such that the sum of the absolute 
deviations from it, i7|x--x^j, is a minimum. 

The median of a discrete set of values needs more 
precise definition. If an odd number of values is ranged 
in monotonic order x^, x^^, ..., so that each 
we shall define the median as the middle value, x^. If 
an even number of values is so arranged as x^, x-^, ..., 
^ 2 n-i» we shall say that the median is any value of x in 
the middle interval, that is, is such that 
The minimal property may then be proved as follows : 

(a) Let there be 2n+l values x^, x^, ..., and let 
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US call the interval between and inclusive the 
interval. The median is at a: = x^- Let us denote by 
i8f(a;) the sum Z\x--Xj\ of absolute deviations from any x. 
First consider S{x) as compared with S{x^), where x 
is in the (ti+I)** interval, on the right of the median, 
and x—Xn == h. Then the absolute deviations of the n+l 
values Xq, a?!, ..., x^ on the one side have each been 
increased by A, while those of the n values a;„^i, a;„^ 2 » 
..., a; 2 n 0^1 fLe other have each been decreased by h. 
Hence in this interval 

S{z)—8(x„) = h. . . (2) 

Now suppose X moves into the next interval, the 
(n+2)**. Comparing S(x) with /Sf(a;„^i), we note that if 
x—Xn+i = h, the absolute deviations of the values 

•••> ^n+i each receive an increment A, while those 
of the remaining n—1 values receive a decrement h. 
Hence in this interval 

S(x)-S{x„^i) = 3h. . . (3) 

In this way S{x) increases as x moves through successive 
intervals to the right, the increments which it receives 
within the intervals being h, Sh, 5h, ..., (2n — l)h; and 
by symmetry, or by a similar proof, S{x) receives corre¬ 
sponding increments as x moves through successive intervals 
to the left of x^- 

Hence /S(a;) is a minimum for x = x^, 

(b) Let there be 2n values Xq, x^, a^gn-i- 
The reader will see at once that if x lies in the central 
interval, the n*^ interval, and if within that interval is 
displaced by an amount h, then n absolute deviations on 
the one side each receive an increment hy while n on the 
other each receive a decrement h. Hence <S(a;) is constant 
within the central interval. 

Also, as X moves out of the central interval either to 
right or to left through successive intervals, S{x) receives 
. o 
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the respective increments 2A, 4A, (2n—2)A. Hence 

8{x) is a minimum within the central interval. 

(c) The result for a continuous variate x can be proved 
as a Umiting case of (a) and (6), or else directly thus : 

Let (—a, b) be the range of values of x, the median being 
taken as the origin x = 0, so that 
ro rb 

I <f)(x)dx = I (l>{x)dx = |. . . (4) 

J -a Jo 


The integral 8{h) of absolute deviations from x = h, 
A>0, is then 


8{h) = r {h—x)(f>(x)dx+ r {x—h)<f)(x)dx 

J -a J h 

= j^J (6) 


whereas 

5f(0) = r — (x)^(x)dx+ f x(f>{x)dx. . 

J~a J 0 

Hence 

8(h)-8(0) = ‘[i: <l>{x)dx—j <^(x)dxj (h--x)(f>{x)dx 
= 2 J (A— x)^(x)dx, . . . ( 


( 6 ) 


and this is essentially positive, since <f>{x) is a positive 
function. The same result may be proved to hold for 
A<0, and so 8(0) is a minimum. 

Note. The indeterminacy of the median of an even number 
of discrete values Xf matters exceedingly little in practice, 
the two middle values being for the most part indistinguishably 
close. 


The Quartiles. The median ordinate halves the 
distribution. Halving again the two halves, we may find 
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values of x which are called the quartile measures. For 
discrete distributions, they lie one-quarter and three- 
quarters along the line of values Xj, supposed arranged 
in ascending order. For continuous distributions of range 
X = a to X — b they are values (it is hardly worth 

while here to press further Greek letters into service) 
such that 

j <l){x)dx = j (f>(x)dx = J. . . (8) 

J a J 

The median might be regarded as a middle quartile g 2 > 
the other two are called the upper and lower quartiles. 
The value of Hq^—qi) furnishes a measure of dispersion 
called the semi-interquartile range. Any value of x has an 
a priori probability of \ of being such that qi<x<q^\ 
it is as likely to be inside the range as outside it. For 
this reason, in the theory of errors, this particular measure 
of dispersion has long been called the probable error of 
the distribution. The name is very misleading, since there 
is nothing specially probable about this particular devia¬ 
tion ; and of late there has been a salutary tendency to 
supersede the so-called probable error by the standard 
deviation, which we now define. 

Standard Deviation. The arithmetic mean of the 
squared deviations from the mean, that is, the 

second moment obviously a suitable measure of 

dispersion. The square root of this, \/^, formerly called 
the root-mean-square deviation, is now called the standard 
deviation and is denoted by <7. The sample value is 
denoted by s. Thus cr^ = /Ltg? = ^ 2 - 

Variance. Modern usage is tending more and more to 
treat itself, rather than cr, as a suitable measure of 

dispersion, under the name of variance. We have therefore 

— Z{x—ii^^j)(x) or j{x—ix\)^<l>(x)dx, . (9) 

while = Z{x—mi)^f{x) .(10) 
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The standard deviation has also a minimal property, 
with respect to the arithmetic mean, namely : 

The sum or mean of squared deviations is a minimum 
when taken with respect to the arithmetic mean. 

This fact is obvious at once from the formula of 14 (5) 

/^2 ~ ^2 

which shows that /Xg can never exceed 

Mean and Variance of Linear Function. If we 
distinguish the respective means and variances of three 
independent variates x, y, z by triple suffixes, thus, 
H'looy H'OiQy H'ooi fi200f H'02 o> Moo2» fhen from the properties 
of cumulants (11) the linear function ax-i-by+cz has 

mean a^tloo+Voio+®/^ooi> 
variance aV 2 oo+*Vo 2 o+‘^Voo 2 ? 
and similarly for a general linear fimction in any number 
of independent variates. 

Range, Extremes. Other indications of the disper¬ 
sion of a distribution are given by the size of the range 
of X itself, 6—a, as well as by the highest value, 6, or 
lowest value, a. 


16. Measures of Asymmetry or Skewness. When 
the mean is taken as origin a; = 0, it may happen that 
^{x) = <f>(—x)l so that the distribution is symmetrical. 

Ex. 1. The distribution of number of heads in a throw of 
n symmetrical coins, described by the g.f. is sym¬ 

metrical about X = \n. 


Ex. 2. The continuous distribution described by 

dp = -^ 

^ V27r 

is symmetrical about x — a. 


Ex. 3. The distribution given 
dp =- 


is symmetrical about a; = 0. 


1 

rr 1 -\-x^ 
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Lack of symmetry, skewness, is revealed functionally 
or numerically in various ways. 

Various Measures of Skewness. In a symmetrical 
distribution the distances of the quartiles and ^3 from 
the median will be equal. In a skew distribution the 
difference between these distances gives a coefficient of 
skewness, namely 

the division by a being for the purpose of removing 
arbitrary units of scale and obtaining an absolute coefficient. 

A natural measure of skewness is however the third 
moment about the mean, fju^. If the distribution is sym¬ 
metrical ^3 = 0 . If the long tail of the distribution is 
on the side of the positive values of x, the cubes of positive 
values of x outweigh the cubes of negative values, so that 
/X 3 is positive, and we have positive skewness. In the 
same way if the long tail of the curve is on the side of 
the negative values of x, then is negative, and we have 
negative skewness. 

To remove arbitrary units of measure, since is of 
the dimensions of x®, or of a®, we construct an absolute 
measure of skewness by dividing ^3 by that is by /jl^\ 
The square of this, is often denoted by jSi- 

Another measure of skewness (due to K. Pearson) 
depends on the fact that in a skew curve the mean, median 
and mode are not the same. The measure in question is 
defined by 

(Mean-~Mode)/(Standard Deviation). 

Like fts it is positive for positive skewness, zero for 
symmetry, negative for negative skewness. 

Skewness of Linear Function. If the 3rd moments 
of independent variates x, y, z about their means are /XgQQ, 
/iosQ, /^oo 3 respectively, the 3rd moment of ax+by+cz about 
its mean is 


^Vsoo+^Voso+^Voos • 
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and similarly for linear functions of any number of in¬ 
dependent variates. This follows (14) because 

17. Measure of Flattening or Excess, Kurtosis. 

Two distributions may have the same mean, the same 
standard deviation, the same skewness, and yet may 
differ in that the curve of the one may be more flattened 
at the centre (platykurtic) than that of the other. 



The degree of flattening is suitably measured by the 
4th moment about the mean, [jl^. Removing arbitrary 
units of measure, just as in the case of jSj, we obtain the 
coefficient often denoted by jSg. It has been observed 

in an extensive class of probability curves, with scale chosen 
so that the variance is unity, that the ordinate at the 
mean or mode is greater or less according as jSg itself is 
greater or less. Thus the value of jSg serves to indicate 
whether the curve is tall and slim at the centre {hptokurtic) 
or squat (platykurtic). In the very important normal 
probability curve, which we shall meet in 32, the value of 

is 3. Hence jSg—3 is sometimes called the excess, curves 
for which j 82<3 being platykurtic, those for which j32>3 
being leptokurtic, the normal curve being taken as 
standard. 

Higher Moments. No simple geometrical inter¬ 
pretation attaches to parameters expressed by moments 
Ur or higher than the 4th, except of course that the 
moments of even order might be regarded as further 
measures of dispersion, and those of odd order as further 
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measures of skewness. These higher moments are in any 
case very seldom used in practice ioi frequency distributions, 
because being computed from values of x liable to random 
irregularity, “ error ” as it is usually called, they may be 
subject to very great error owing to the raising of some 
abnormally frequent large deviation x to a high power. 
This will be apparent when we come to consider the 
sampling error of coefficients, in Chapter VII. 

18. Practical Computation of Moments. The 

initial stages of the analysis of frequency distributions 
almost always involve the computation of ordinary or 
factorial moments. In the case of a continuous variate 
artificially grouped (13) into classes, a certain error is 
introduced into the moments by the centring of class- 
jfrequencies about the centre of the class. The calculated 
moments then require adjustment by formulas of rather 
wide application called Sheppard*s Corrections. 

The example on page 40 shows the computation of the 
first four moments and the coefficients of dispersion and 
excess, for a frequency distribution. The column headings 
explain themselves. It will be observed that transference 
is made to the more convenient provisional mean x = 67, 
this being judged by inspection of the distribution to be 
somewhere near the true mean. 

Sheppard’s corrections have not been used ; we shall 
allude to this example when we come to discuss them. 
As for the mean height of the group, the provisional origin 
is really, as we saw earlier, 67 tV, or 67-44 inches. Hence 
the mean height is 67-44+0-34 = 67-78 inches. 

The distribution shows a slight negative skewness. 
Whether this is a genuine effect or due to the irregularities 
of sampling cannot be decided until we know more about 
the probabihty distributions of coefficients calculated from 
samples. 

The reader should verify that the sample estimates of 
and P 2 0-0014 and 3-56. 
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Example. The distribution of heights of adult Irishmen. 


X 

nf 

X 

nfx 

n/aj* 

nfx^ 

n/aj* 

69 

1 

-8 

-8 

64 

-512 

4096 

60 

0 

-7 

0 

0 

0 

0 

61 

2 

-6 

-12 

72 

-432 

2692 

62 

2 

-6 

-10 

60 

-250 

1260 

63 

7 

-4 

-28 

112 

-448 

1792 

64 

15 

-3 

-45 

135 

-405 

1215 

65 

33 

-2 

-66 

132 

-264 

528 

66 

58 

-1 

-68 

58 

-58 

58 

->67 

73 

0 ( 

-227) 0 

0 

(-2369) 0 

0 

68 

62 

1 

62 

62 

62 

62 

69 

40 

2 

80 

160 

320 

640 

70 

25 

3 

75 

225 

676 

2025 

71 

16 

4 

60 

240 

960 

3840 

72 

10 

5 

60 

260 

1250 

6250 

73 

3 

6 

18 

108 

648 

3888 

346 

' = 0-341+67. 

, = 4-821-(0-341)2 

(345) 

)118 

0-341 

= 4-706. 

)1668 

4-821 

(3916) 

)1546 

4-468 

)28236 

81-61 


wij = 4-468-3(0'34l)(4*821)H-2(0-341)» = -0-385. 

m4 =- 81-61-4(0-341){4-468)+6(0-341)2(4-821)-3(0*341)* =78-84. 


19. Computation of Moments by Repeated Sum¬ 
mation. If the origin of a distribution be taken at either 
* end, preferably at the lower end, factorial moments can 
be computed by a process of repeated summation. We 
sum frequencies in columns from the remote value of x 
towards the origin, in the manner exemplified below. 
The leading sum in each column is one step lower than the 
leading sum in the preceding column. 
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Ex. 1. The same distribution. 

with origin at x 

= 69. 

X 

nf 

E 


E^ 

Z* 


59 -f- 0 

1 

346 





1 

0 

345 

2886 




2 

2 

345 

2541 

11407 



3 

2 

343 

2196 

8866 

28343 


4 

7 

341 

1853 

6670 

19477 

49757 

5 

15 

334 

1512 

4817 

12807 

30280 

6 

33 

319 

1178 

3305 

7990 

17473 

7 

58 

286 

859 

2127 

4685 

9483 

8 

73 

228 
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1268 

2558 

4798 

9 

62 

155 

345 

696 

1290 

2240 

10 

40 

93 

190 

350 

595 

950 

11 

25 

53 

97 

160 

245 

355 

12 

15 

28 

44 

63 

85 

no 

13 

10 

13 

16 

19 

22 

25 

14 

3 

3 

3 

3 

3 

3 


r I 

1 

1 

2 

6 

24 


The successive sums at the heads of columns may be proved 
(Appendix 2) to be equal to We have therefore 

n == 346, = 2886, ~ 22814, = 170058, 

nm' . — 1194168. 

( 4 ) 

Transforming to ordinary moments m' by the relations 
(Appendix 3) 

m; ^ m;,j, 

= ’”(2)+Sl)’. 

K = >) + 3”»(2)+Si)’ 

= ”"(4) + ®S8) + ’^”‘(2)+'”a)’ 
we obtain = 2886/346 = 8*34104, 
mg = 25700/346 =74-2775, 
mg = 241386/346 = 697-647, 
m^ = 2377100/346 =6870-23, 

from which, by adjusting (14 (5)) to moments about the 
mean, we derive 

ma = 4-706, m, - —0-386, = 78-87. 
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Now it may be noted that some advantage has been 
lost through the large numbers that arise in summations 
from end to end. Even though six significant digits have 
been retained throughout, the final results are very slightly 
discrepant with those computed by the other method. To 
obviate these disadvantages (which are not serious when a 
calculating machine is available) one may use either (i) fac¬ 
torial moments obtained by summation from both ends 
towards an origin near the centre, or (ii) central and mean 
central factorial moments obtained by a slight modification 
of this summation. 

Ex. 2. Ordinary factorial moments. Origin at a; = 67. 



S 
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27* 
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1 

1 

1 

1 

1 

1 

0 

1 

2 

3 

4 

6 

2 

3 

6 

8 

12 

17 

2 

6 

10 

18 

30 

47 

7 

12 

22 

40 

70 
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16 

27 

49 

89 
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33 

60 
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68 
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227 
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1415 

73 
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62 

165 
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40 

93 
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350 



26 

63 

97 
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245 


16 

28 

44 

63 

85 

no 

10 

13 

16 

19 

22 

26 

3 

3 

3 

3 

3 

3 

r! 

1 

1 

2 

6 

24 


From the italicized entries we obtain 
n = 228+118 = 346, 
nm'i) = 346-227 = 118, 

= (360+426)2 = 1660, 
nm = (246-782)6 = -3222, 
nm^^) = (110+1416)24 = 36600. 
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These may be transformed to ordinary moments m' by 
the same relations as before, yielding 

m' = 118/346 = 0-341, = 1668/346 = 4-821, 

= 1646/346 = 4-468, m\ = 28236/346 = 81-61. 

These are the same values as were found by the first method. 

Ex. 3. Central and mean central factorial moments, with 
the same origin x = 67. 

Here we again sum towards the centre from the ends, but 
each alternate sum (shown bracketed and italicized) involves 
the adding of only half the last summand in the preceding 
colunm, while the last sums in the other columns step 
successively away from the centre, as shown. 


nf 

^ - 
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86 
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22 

25 

3 

3 

3 

3 

3 

3 
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r! 1 

1 

2 

6 

24 


From the italicized entries we obtain the central factorial 
moments 

n = 191-6 + 164-6 = 346, 
nm'jj = 345-227 = 118, 
nm' = (622-5+311-6)2 = 1668, 
nm = (695-367)6 = 1428, 
nm|^) = (662-6+464-6)24 = 26668. 
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The formulas for the m' in terms of the are rather 
simple (Appendix 3). We have 

= m' = 118/346 = 0*341, 
mg = m = 1668/346 = 4*821, 
mg = wjgj+m' = (1428 + 118)/346 = 4*468, 
fn\ = = (26668+ 1668)/346 = 81*61, 

as before. The moments about the mean can now be found. 
Alternatively, we could easily derive formulae transforming the 
moments and transferring them to the mean in one step. 

20. Sheppard’s Corrections for Grouped Moments. 
As mentioned earlier, when a continuous distribution has 
been grouped into centred classes for convenience, the 
moments require adjustment or correction because of this 
artificial grouping. The necessary formulae of correction 
were foxmd by W. F. Sheppard. 

Naturally the problem for perfectly general functions 
<f)(x) is too broad, and it is necessary to impose conditions. 
Sheppard considered the case where <f>(x) was such that 
the derivatives ^"(x), ... vanished in succession at 

the boundaries x = a and a; = 6 to such an order that 

J x'^(f>^*^{x)dx = wljx'^j<f>^^^{x^) . . (1) 

to a suflGlcient degree of accuracy, where w is the class- 
breadth and Xf the centre of a typical class ; that is to 
say, the error committed should be negligible compared 
with sampling errors. 

Remark, The relation between an. integral and a sum of 
equidistant ordinates of the kind here considered enters into 
pure mathematics in the Euler-Maclaurin summation formula^ 
by which a sum of ordinates is expressed as an integral over 
the range plus corrective terms involving the derivatives of 
odd order taken at the boundaries. In many cases, where 
the derivatives ^'( 05 ), ... are not absolutely zero but 

converge to zero as a limit, the representation of the integral 
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on the left of (1) by the sum on the right needs very careful 
investigation. It is found, however, that for the statistical 
functions to which Sheppard’s corrections are usually applied 
the difference between the integral and the sum can be made 
J^®gligibly small by taking values of the class-interval w oi o, 
size quite customary in practice. Usually it is enough for w 
to be less than the standard deviation. The derivation of 
the formulas given below must be regarded as approximate 
only. 

Ex. 1. The following two comparisons of integral with 
sum over an infinite range are interesting in this respect: 


whereas 



TT — 3*14159 nearly. 


00 1 

Z —-Ty = ^ coth TT = 3-15334R... 


X taking the values 0, ±1, ±2, ... . Again, 

V'Zir — 2*506628275 nearly, 


/: 


whereas 


E e-1** = 2*506628288 nearly. 


X taking the same values as before. The first sum in these 
examples is only moderately close to the corresponding 
integral; the second is very close, and still closer results are 
obtained if a summation with a finer subdivision of x is used. 


Suppose the range 6—a divided into n class-intervals 
{x—\Wy x-{-\w)y so that 6—a = nw. If the probability in 
the class is 



then the r** moment calculated from the grouped classes is 


(if — U XyPf, . . . ( 3 ) 

i“i 
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whereas the true moment is 

n 

Now 

rxi+\w 


== J x^<l>{x)dx. 


Pi 


r*y- 
J Xf- 


<f){x)dx 


kU) 




= r fj}{Xi'\-x)dx 

J —\w 

= f {<i>{Xi)+x^'(x,)+x^''(Xj)j2\-ir...)dx 

J -Jw 

= w<f>{Xj) +tci^<l>’{xi)l24:+w^ifP{Xj)ll920 +..., 


( 4 ) 


( 6 ) 


provided that this series in powers oftirconverges. 

Hence 

ft’, = 

= jx^<l>(x)dx+^jx^<l>’(x)dx+^^jx^,l>^{x)dx+..., ( 6 ) 

in view of (1). Integrating by parts and using the fact 
that derivatives vanish at the boundaries, we have 
tv ^ f tv^ f 


1920 


If moments /x^ about the mean are taken, we have 
therefore the relations (where means the moment of 
the grouped classes about the mean): 

/^o = Mo = 1. 

Ml = Ml == 0. 

/4 = Ms+j^^'Vo = M2 +i4 “'** • • (8) 

Ms = M 3 + 4 *" Vi = 

Mi =M4+^«’V2+^«’*. - 



SHEPPARD’S CORRECTIONS FOR GROUPING 


47 


whicli, on being solved for the /x,., yield 
== fh = 0 . 

Ih = tH- • • • • ( 9 ) 

H'S ^ /^ 3 > 

1,1 

/^4 = 2"’V2 + 


and these are the required adjustments, Sheppard's 
corrections. The correction to the second moment is 
especially simple and noteworthy. If the class-interval 
is taken as the unit of scale, the correction amounts to 
subtracting from the grouped second moment. 

It is customary, though the practice requires more 
justification than it has ever received, to apply the same 
corrections for grouping in the case of frequency distri¬ 
butions, the presumption being that the moments thus 
corrected are a better representation of the moments of 
the underlying probability distribution. 

Ex. 2. Correcting the moments about the mean for 
grouping in the example of 18 and 19, we obtain for the 
corrected moments 

mo - 4-705-0*083 = 4-622. 

mg - --0*386. 

^4 = 78*84-J(4-705)+0-029 == 76-62. 

Ex. 3. The reader should seek out for himself numerous 
examples of frequency distributions, and should acquire as 
much practice as possible in computing moments in the various 
ways exemplified above, and in correcting them. Sheppard’s 
correction will be applied in those cases in which the relative 
frequency f(x) in sample corresponds to probability ^(a;) of a 
continuous variate. 
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SPECIAL PROBABILITY DISTRIBUTIONS 

21. Distributions of Equal Probability. If n values 
of Xy where j = 1 , 2 , ti, have each equal probability 
Ifriy the graph of probability consists of n ordinates of 
equal height 1 /n. The case of a symmetrical coin is the 
case n = 2 , the case of an ordinary unbiassed die is the 
case 71 = 6. ——' 

The Rectangular Distribution. The limiting case 
of the preceding, when n tends to infinity, yields an 
important distribution called the rectangular distrihutiony 
namely that in which x has an equal probability of being 
at any point in the range x = a to x = by a<b. The 
probability differential is then given by 

dp = dXy ... (1) 

0 —a 

so that (f>{x) = 1 /( 6 —a) and the probability curve consists 
of a rectangle on the range as base and of height 1 /( 6 —a). 
It is always possible to choose the central point of the 
range for origin, and the unit of scale such that the range 
becomes the new range x = — J to x = The rectangle 
is then a square. The moments of odd order vanish; 
those of oven order are 

j x^dx = (i)^’ • • (2) 

In particular /Lt 2 = ^ = l/V^^ = 0*2886... 

Example 1. Show the m.g.f. of the standard rectangular 
distribution is (sinh Ja) Ja. 

48 
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Example 2. The following samples from a rectangular 
population have been arranged as frequency distributions. 
The times on 1000 watches displayed in watchmakers’ 
windows were noted by the author. The distributions are 
of the first and second 600 of these. Class x = 1 means 
the class of all watch times from 1 h. to 1 h. 69 m. to the 
nearest minute, and classes a? = 2, 3, ... 12 have a similar 
meaning. 



X 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

n 

(i) 


34 

54 

39 

49 

45 

41 

33 

37 

41 

47 

39 

41 

500 

(ii) 

n/ 

47 

41 

47 

49 

45 

32 

37 

40 

41 

37 

48 

36 

500 


The mathematical expectation of the number in any class 
is 500/12, or 42 to the nearest integer. One of the classes in 
the above samples contains 64, and another contains 32. We 
shall see later that the deviations here are not extreme. 

The mathematical expectation, or mean of x in the 
population, is 6*5. The means of x in the above samples 
are 6*426 and 6*322. 

22. The Binomial Distribution. This fundamental 
distribution arises when n trials are made of a constant 
system S with probability p of an event E, the number x 
of successes in the n trials being the variate. The g.f. 
is {pt+q)^, and so by binomial expansion the probability 
function, namely the coefficient of in the g.f., is 

<f>ix) = . . (1) 

Moments. The f.m.g.f., obtained by putting ^ = 14-a, 
is seen at once to bo (1+j^a)”, so that the mean, the 
coefficient of a in this, is np. Hence the f.m.g.f. about 
the mean is (11) 

(l+a)-”^(l+i>a)’» 

= [{l+pa)(l—pa+p(p+l)a^l2\—p(p+l)(p+2)a^l^\-\^.,,)]n 

= [l+jP(l—i’)“V2!-2jj(l-p)(p+l)aS/3!. (2) 

whence |a(2) = i^o) = —2wp^(p+l), so that 

/ig = = npq, 1 x 3 = /i(3)+3/t(a) = npq(q-p). (3) 

• D 
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It is readily proved in the same way, by finding and 
hence fjL^y that 

/X 4 = npq[p^+(3n—4)pq+q^]. . . (4) 

The formula a = fundamental importance. 

Example. The following is a sample from a binomial 
population. The Swedish astronomer and statistician, C. V. L. 
Charlier, performed 1000 times the experiment of drawing 10 
cards, one at a time with replacement after each drawing, 
from an ordinary pack, the number x of black cards in each 
set of 10 cards being the variate. Thus n = 10, p = J. He 
obtained the distribution 

a;0123 4 6 6 789 10 iV 

Nf 3 10 43 116 221 247 202 116 34 9 0 1000 

The corresponding probability distribution has g.f. 

Multiplying this by 1000 and recording the 
coefficients of powers of t to the nearest integer, we obtain 

a?0123 4 6 6 789 10 AT 

N4> 1 10 44 117 206 246 206 117 44 10 1 1000 

From Charlier’s data we find = 4*933, = 2*415. 

The theoretical expectations are = np — 5*00 and 
jLta = npq = 2*5. 

We shall consider in a later section (55) whether these 
deviations of actual experimental results from theoretical 
expectation are reasonable under the hypothesis of random 
sampling. 

23. The Binomial Distribution of Poisson. The 
ordinary binomial distribution is often called the 
BemouUian distribution, after James Bernoulli, who first 
(in Ars Conjectandi, a work published in 1713, eight years 
after his death) investigated it in detail. S. D. Poisson 
in 1837 considered the problem of n trials, but with the 
system S varied each time so as to produce possibly 
different probabilities of success p^, where J = 1, 2 , ..., n. 
The g.f. is therefore 

iPii+qiHPit+gt) — ipJ+qJ, • • (i) 
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and so the f.m.g.f. is 

(l4-^?ia)(l+^?2a) ... • • • (2) 

The coefficient of a in this f.m.g.f. gives us the mean, or 
mathematical expectation of the number of successes, as 
Pi +• • • +Pn* write 

np = Pi+P2+--+Pny • • (3) 

in order that we may later compare the moments with 
those of a Bernoullian distribution with the same mean 
probability p, and so characterized by the g.f. (pt+q)^. 
The Poisson f.m.g.f. about the mean is (compare the details 
of 22 (2)) 

i7( 1 1 -]-Pja) (n = product) 

5 

— \+Zpiqja.^l2\—2Zp^j(Pi+\)a?l‘6\^-... . (4) 

Hence /X(2, = = Zpjq„ and /i,,, = —2Zpiq,(pf^\), 

so that =liiz)+ip^ 2 ^=Zpi(ij{qi-pi). . . (5) 

24. Comparison of Bernoullian and Poissonian 
Variance. It will now be proved that the Poissonian 
variance, let us say (t|„ is less than the Bernoullian, 

At first sight this may seem surprising, for one might 
imagine that the variation of probability of success in 
trials within the experiment would increase the variance 
of «, the number of successes. If we consider, however, 
the case of extreme variation of probability, namely the 
case in which some of the trials are certain of success, 
and the rest are certain of failure, we shall see that the 
smaller variance is natural enough ; for in this extreme 
instance the value of x is constant and so its variance is 
zero. 

The fact that the Poissonian variance is less than the 
Bernoullian is valuable, for it suggests a test for the 
constancy oi: otherwise of the system 8 from one trial 
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to the next, in other words, for statistical homogeneity 
within the experiment. 

As in 23, let p be the mean probability, p — Ep^jn, 
We have at once, by the usual transference to the mean, 

ci^ = ZiPf-pfIn = Zp^ln—p^, . . ( 1 ) 

where is the variance of probability in the n trials. 
Hence 

Spfq,=Zp,(\-pi) 

= np—np^—E(pj—p)^ 

= npq-I{pj-p)^ . . . ( 2 ) 

that is, < 7 |» = a|—. . . • (3) 

This result shows not only that the Poissonian variance 
is less than the Bernoullian, but by how much it is less. 

25. The Lexian Distribution. The extension made 
by Poisson to the Bernoullian scheme consisted in varying 
the probability of success among the n trials, but within 
the experiment. A different kind of extension was con¬ 
sidered by the German economist, W. Lexis, in 1877. 
The probability was taken by Lexis as constant in the 
n trials of one experiment, but as varying among k such 
experiments. 

Let k Bernoullian sets of n repeated trials be made, 
each with constant probability of success within the set. 
Let Pi be the probability for the i*^ set, where i = 1 , 
2 , ..., k, and let Xi be the number of successes recorded 
in it. It is required to find the mean and variance of 
the distribution of the 

The sets are here mutually exclusive, and the probability 
of each, if we imagine one of the Pi to be chosen and n 
trials to be then made, is Ijk, Also the f.m.g.f. of is 
(l+jj^a)". Thus the f.m.g.f. of the Lexian distribution is 

kr^E(l+Piar. . . . (1) 

i 

The coefficient of a shows that the mean is nEpfk, For 



LEXIAN DISTRIBUTION 


53 


comparison with a repeated Bernoullian scheme let us put 
np = nZpJk, The f.m.g.f. about the mean is then 

k-\l+a)-^^E(l+Pia)^ .(2) 

= [\--npa-\-np(np + \)a^j2\-\-,..] 

X [1 +npa + — 2/2! +...] 

whence, by picking out the coefficient of a 2/21, 

= /^(2) 

= np (np +1) — 2?i 2p 2 —l)k~^[kp^ +^(Pi —p) 

= npq+n(n—l)E{pi~pyik, 
that is, 

(t| = npq+n(n — l)al, . . . (3) 

Thus, whereas the Poissonian variance was less than 
the Bernoullian, we see that the Lexian variance exceeds 
the Bernoullian by an amount which increases strongly 
with n, because of the coefficient n{n—\) in (3). 

26. Goolidge’s Extension of the Lexian Scheme. 

It is a natural extension to consider, as J. L. Coolidge 
did in 1921, the distribution which arises not from k 
Bernoullian but from k Poissonian sets, each with a different 
set of probabilities in its constituent n trials. 

Let Pif be the probability of success in the trial of 
the i** set. Then, just as in 25, the f.m.g.f. is 

k-^En{l+Piia), . (1) 

i i 

Let us write Ep^^ = Epi^ = kp. Then the mean of 
i i 

the distribution is evidently np. Transferring the f.m.g.f. 
to the mean, and picking out the coefficient of a 2/21, 
wo find, after three or four lines of algebra, 

= M(2) = npq+n{n-\)E(Pi^-pYlk-ZE{pi}-pt^)^lle. 

. i i i 
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It is appropriate to regard the three terms of this 
expression as of Bernoullian, Lexian and Poissonian type 
respectively. Certain special cases are easily perceived ; 
for example, when = p, that is to say, when the mean 
probability in each set of trials is the same for all sets, 
a variance emerges which slightly generalises the Poissonian 
variance af,, and, like it, is less than the Bernoullian. 

An alternative form is 

/Xj = npq+n^2:{j)io-p)*lk-2JS{pti-p)»lk, 
i i j 

which we may write as 

al,= al+n^ol-nol. . . . ( 2 ) 

This result shows that non-hornogeneity, or fluctuation 
of probability, within the trials of an experiment is of far 
less effect, when n is large, than fluctuation in mean 
probability from one set to another. In fact in many 
cases differs only slightly from the corresponding (t|. 

Analysis of Variance. The results which we have 
obtained for the Lexian and Coolidge schemes exhibit the 
variance as resolved into separate components of variance. 
The Bernoullian component may be called the random 
component, since it arises even when probability is con¬ 
stant, while the Lexian component may be called the 
systematic component, since it arises from the systematic 
alteration or variation of probability from one experiment 
to another. This resolution of variance into separate 
components of variance has been called analysis of variance. 
It has been greatly extended by Professor K. A. Pislier , who 
has devised regular schemes of experimental arrangement 
involving many variates, by means of which not one but 
several systematic components of variance can be isolated 
(75) from each other and from the random component. 

27. Gharlier’s Criteria of Homog’eneity Based on 
Dispersion. The test of homogeneity or stability con¬ 
sidered in this section would now be superseded or 



VARIATION IN PROBABILITY 


55 


amplified by modern methods of analysis of variance, 
but it is interesting in itself. 

We have approximately, in the Lexian and Coolidge 
schemes, 

o| = a%+n^al. . . . (1) 

Hence (ajpf = ■ • • (2) 

where ri'p, the mean of the distribution. 

Hence . . (3) 

Charlier denoted this by p, naming it the “ coefficient of 
perturbation of a Lexian distribution. He turned it 
into a percentage by taking lOOp. From (3) we see that 
p measures the relative fluctuation of probability. 

Example. Classing 288,000 Swedish births in 576 sets 
of 600 each, according to different months and different 
districts, Charlier found for x, the number of male births 
in a set, 

m\ = 257*12, = 12*49, n = 600, k = 676. 

Hence p=mi/n ==0*514, q — 0*486, not a priori^ but as 
estimated from the large sample of 288,000 ; and so 

s^ = ^/(npq) = V124-9 = 11*18. 

Hence lOOp = 100(156*0-124*9)/257 = 2*17 per cent. 

The conclusion made is that a male birth in Sweden is an 
event of 61*4 per cent, probability, with a standard deviation 
of 61*4x0*0217, or about 1*1 per cent, probability. 

28. Types of Multinomial Distribution. The bino¬ 
mial distribution, of BernouUian or Poissonian type, is a 
special case of the multinomial distribution, the forms of 
which are so many and so various as almost to defeat 
classification. We have seen a simple example in the 
probability distribution of totals of points in n throws of a 
die, or a single throw of n similar dice. Here the g.f., for 
biassed dice, is 

{Pit+P4^+Pif^+Pj'*+Pj'^+P«^)''' ■ ■ ( 1 ) 

and it is best to leave the distribution in this symbolized 
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form, and not to expand by the multinomial theorem. 
The generalization to the case of n different dice, possibly 
with different numbers of faces, is easily seen. 


Ex. 1. Prove that the mean value of the total in n throws 
of a biassed die is n(pi+ 2 pa+ 3 p 8 + 4 i ?4 + Si ^6 + 6 ^? 5 ). 

Ex. 2. Find, by constructing the f.m.g.f., the variance 
and standard deviation of the total of points in a throw of 
n symmetrical six-sided dice. 

The f.m.g.f. reduces to 






Hence f/g - 35n/12, and so a = \/(35n)/2 V3. 


29. Sampling without Replacement, Hypergeo¬ 
metric Distribution. When in sampling a population the 
individual drawn is not replaced, the result of one drawing 
influences the probability of the next, so that the successive 
drawings are not independent. Hence it is no longer 
possible to combine into a product the g.f.’s of the separate 
drawings. It is true that the difficulty can be circum¬ 
vented by the introduction of symbolic products, with 
due precautions in expansion, but we shall here proceed 
from first principles. 

Let us consider a population of N individuals, of whom 
M = Np are of character A, so that the probability of 
drawing an A at the first drawing is p. Let n drawings 
be made, no individual drawn being replaced after the 
drawing. It is required to find the probability distribution 
of X, the number of individuals A drawn. 

The probability of x successes A, n—x failures 
occurring in some particular order, is 

... {M-x+l)[N-M){N-M-l) ... 

{N-M-n+x+l)IN(N-\) ... (i\r-w+l), . ( 1 ) 
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as is readily seen by considering how the numbers in 
population, and in categories A or A, are depleted by 1 
at each drawing. But there are n^x) possible orders in 
which X successes may eventuate among n drawings. 
Hence the desired probability is 

^(x) - . ( 2 ) 

where = M(M—l){M—2) ... {M—x+l), and so on. 

Just as the binomial probability function of 22 was a 
typical term in the binomial expansion of ('pt+q)'^, so this 
function that we have just found is a typical term in a 
certain series, a hypergeometric series. Hence <j)(x) is often 
called the hypergeometric probability function. 

The g.f. is the hypergeometric series 

E . (3) 

j|f(n) 

N-^m-n+l ; t) 


in the notation of Gauss, and so the f.m.g.f. is 

x = 0 


which may be evaluated (by gathering terms together in 
Vandermondian expansions) as 


1+-^ a+ —^-( 2 ,— a /2!+ « /3!+..., 


( 5 ) 


= F {—M, —TO ; —N ; a). 


a terminating hypergeometric series. The mean is thus 
MnjNy and the r^^ factorial moment is 

The examples which have now been given of probability 
distributions have shown how numerous and varied are 
the types of distribution. In fact, any proposed probability 
function may be simulated by a suitably constructed model 
or population, and special samplings of this population 
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give rise to further probability functions. Fortunately, 
when the number n of trials is large, many of these pro¬ 
bability distributions tend with good approximation 
towards one or other of a few dominant tjrpes, which we 
shall now consider. 

30. Important Approximate Distributions : Types 

A and B. When the coefiScients of in the Bernoullian 
binomial g.f. are taken as probability ordinates 

y = j>{x)y we may join the tops of the ordinates to form 
a probability polygon. If this is done for increasing 
values of w, the mean np being taken as origin and the 
standard deviation 's/inpq) as unit of scale, it is found 
that the successive probability polygons tend to lose any 
initial asymmetry due to inequality of p and q. 

In fact the coefficient of skewness is 

ft = mI/mI = inpq(q-'p)fl{n'pqf 

= • • • ( 1 ) 

which evidently tends to zero as n increases, unless either 
of p or g is of the order of magnitude of 1/n, let us say 
0 (l/?i), in which case the skewness remains appreciable. 
Not only so but, apart from the exception just mentioned, 
these binomial curves are foimd to cluster towards a 
limiting symmetrical shape, the same for all. The curve 
to which they thus approach asymptotically is of paramount 
importance in statistics, and is called the normal probability 
curve. It is the asymptotic shape not merely of the 
Bernoullian binomial but of the Poissonian, as well as of 
the multinomial and of many other distributions, and it is 
characterized by the probability differential 

dp = 4,{x^x = . . (2) 

aV27r 

where p, is the mean, a the standard deviation. 

When small corrective terms involving n are retained, 
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a closer representation is given by the probability function 
of Type A, namely 

p[x) = "(a;)/3!+a4<^i^(a:)/4!-..., . (3) 

where <f>{x) denotes the normal probability function, and 
the coefficients of the derivatives ^^^^{x)lr\ are of 
irregularly decreasing orders of magnitude with respect to 
n. The coefficient a^, when freed of arbitrary units, 
measures skewness, measures excess. 

As noted above, the case when p is very small is 
exceptional. If p is 0(\jn), the mean np is not 0(n) 
but 0(1). In this case the normal function is not the 
most suitable basis of approximation, and the appropriate 
asymptotic probability function is Poisson's function of 
statistical rareness, namely 

^(x) == . . . (4) 

where p, is the mean. Here again, when terms of smaller 
order involving n are retained, a closer representation is 
given by the probability function of Type B, namely 

p(x) = (^(a;)+62vW/2!-63V^^(^)/3!+..., . (5) 

where ^(x) is Poisson’s function (4) above, and V denotes 
the operation of forming the receding difference, so that 
S7i/j(x) == iIj(x)—i/j{x—1 ). It proves to be the case that 
62 is 63 and are Oin-'^), 65 and are 0 (n-^), 

and so on. 

We now consider the derivation of these functions. 

31. The Normal Fimction as Limit of the Binomial. 

The rigorous derivation of the normal function as generated 
by compounding n independent distributions, and the 
discussion of necessary and sufficient conditions, require 
advanced mathematics beyond our scope. We content 
ourselves here with elementary and incomplete treatments. 

Consider first the binomial g.f. (pt+q)^, where p is 



60 


SPECIAL PROBABILITY DISTRIBUTIONS 


not of order l/n, but is 0(1). Putting i = e® we have 
the m.g.f. 

{'pe^+q)^ = (l+pa+pa2/2!+pa^/3I+)”.(1) 

The mean is np. Let us transfer to the mean, and to 
discover the limiting shape of the curve of probability 
let us alter the scale, so as to jSnd the distribution, not of 
actual number of successes a, but of the deviation 
{x—np)ln of the relative frequency of successes from the 
mean p of relative frequency. 

As a first step we construct the m.g.f. of xjn. By 
11 it is 

[1 -\-pajn+pa^j2\n^-\-0(n~'^)Y 

= [(l+pa/7i+ipV/n2)(l+i(p-p2)a2/^2 (2) 

where 0 (n~‘^) indicates in both cases remainder terms of 
order . As n increases this m .g.f. tends asymptotically to 

gpagi;:gaVn.^3^ 

The first factor shows that the mean of the transformed 

variate is p ; but this we already know. The second 
factor indicates, by a further obvious transformation of 
scale, that the m.g.f. of the standardized deviation 
z == {X’-np)l^/{npq) is 

.(4) 

Now the possible number x of successes may range from 
0 to n. Thus the values of z may range from — y/(nplq) 
to ^ range which tends in both directions to 

infinity. Further, consecutive values of x differ by 1, 
and so consecutive values of z differ by ^jy/inpq), an 
interval which tends to zero as n increases. We therefore 
seek a representation of the probability function ^(2) as 
a positive function continuous over the range —00 to 00 ; 
and the question is, what function <f)(z) is such that its 
m.g.f. 


^z)e^^dz = ? 

—00 


. ( 5 ) 
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The answer is contained in a theorem, to the effect that 
the only positive continuous function satisfying this 
relation for some continuous range of values of a is 

^(2) =. . . (6) 

V2-7T 

and this is the normal probability function, in standard 
form. 

The reader should become thoroughly familiar both with 
this form and with the unstandardized form of 30 (2). 

Incidentally, taking the logarithm of the m.g.f. (4), 
we see that apart from the mean or first cumulant there 
is only one other cumulant, namely /cg or 


32. Properties of the Normal Probability Function. 

The curve of the normal function is a symmetrical bell¬ 
shaped curve, extending to infinity on either aide and 
flattening rapidly upon the axis of x. 



The maximum ordinate is = l/\/(27r). 
under the curve is 


1 

V 277 



= 1 , . 


The area 


. ( 1 ) 


by the well-known integral. (Gillespie, Integration, p. 88.) 
The points of inflexion, given by d^yjdz- = 0, will be 
found to be at 2 = or, in unstandardized units, at 
deviations ia from the centre. 

The probability, as taken from the normal curve, that 
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a deviation from the mean is numerically less than z is 
the area under the curve between the ordinates for —2 
and + 2 , namely 


1 

'\/2'rT 


/: 




. ( 2 ) 


This function, called the error function or probability 
integral^ is denoted by erf( 2 ) and has been extensively tabu¬ 
lated. (It is called the error function because the typical 
distribution of errors committed by instruments of observa¬ 
tion has been found to be sensibly normal.) The following 
short table shows how the probability of deviations outside 
the range (—z, 2 ) diminishes as z increases : 

z 0 0*6 1-0 1-6 2 0 2*6 3*0 3-6 4-0 

erf(z) 0 0-383 0-683 0-866 0-954 0-988 0-997 0-9995 0-99994 

We may note that the probability of a deviation 
greater than o is about 1/3 or more nearly 7/22 ; that 
of one greater than 2(7 is about 1/20 or more nearly 1/22 ; 
that of one greater than 3c7 is about 1/370 ; and that of 
one greater than 4c7 is about 1/17000. 

The quartile deviation or so-called “ probable error ** 
is given by 

■ ■ . ( 3 ) 

By interpolation it is found to be 2 = 0-6745 nearly, 
corresponding to a deviation from the mean of about 2/3 
or more nearly 27/40 of the standard deviation. 

The mean absolute deviation is given by 

-^=. f ze~^^*dz = = 0-7979 nearly, . (4) 

V 27 r J 0 

corresponding to about 4/5 of the standard deviation. 

The higher moments of the normal fimction are found 
by expanding the m.g.f. exp(^a^), or the unstandardized 
exp(J< 72 a 2 ), and observing the coefficients of d^/rl. For 
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odd orders they vanish, for even orders 2 r they are given by 


In particular 


Mar = (i)^a2-(2r)!/rl 
M4 = SaS 


. (5) 


so that (17) the coefficient of excess jSg = M4/M2 ” 


33. Poissonian Function of Hare Statistical 
Frequency. We return to the binomial g.f. ipt+q)^y 
examining the previously excepted case in which, though 
n becomes large, p is so small that the mean np is 0(1) ; 
in fact p = 0(71“^). Writing the mean np as /x, we have 

The f.m.g.f. is therefore (22) 

(l+/xa/n)", which tends to . • (1) 

as n increases- This is the f.m.g.f. of the Poissonian 
function. The probability g.f. is therefore 

.( 2 ) 

and the coefficient of <* in this gives the desired probability 
function as 

ijj(x) = e-^fjL^lxl . . . (3) 

34. Properties of the Poissonian Function. The 
normal function contains two parameters, the mean ^ 
and the standard deviation cr. The Poissonian function 
has one parameter only, the mean /x. The range of the 
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function is from x = 0 to x= oo. For fi<l the pro¬ 
bability polygon is J-shaped, for jLc>l it becomes double¬ 
sided and for large values of fi tends to acquire symmetry. 
Indeed, for large values of [jl the shape is approximately 
normal; for the ordinary m.g.f. is 

expDi(e®—1)] = exp(jLW3t+/ia^/2!+jLta®/3!+...) (1) 

and if we change the scale so as to make y/fi the unit we 
obtain the g.f. 

exp(/ti*a+a*/2!+a*/3!^‘+...). • • (2) 

which, to a first approximation (that is, including the 
first two terms of the series in the bracket) is the m.g.f. 
of a normal function with mean \/fjL and unit standard 
deviation. 

The logarithm of (1) gives the cumulant g.f. of the 
Poissonian function as /x(e“—1), which shows that all the 
seminvariants are equal to the mean [x ; in particular 
the variance /cg or /ag is equal to /x. 

There is only one factorial cumulant, /C(i) = /x. 

35. More General Derivations ; Types A and B. 

As before, the extensions of the domain of application of 
the fundamental distributions given below are not 
established under the widest conditions. 

Let us consider the compounding of n systems Sj, 
where j = 1, 2, ..., w, where each system has finite cumu- 
lants, all 0(1). The cumulant g.f. of 8 ^ is then a convergent 
series 

K^{a) = K<^a+Kiyi2\-^K^a^l3\ + .(1) 

For example, the binomial distribution of n throws of a 
coin, provided that p is 0(1) and not 0(ri“^), may be proved 
to have a c.g.f. of this kind. 

Now imagine all the n systems 8 j to operate independ¬ 
ently, the results being added to make a variate x. By 
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the additive property of cumulants the cumulant g.f. 
of X is 

i;(/c<%+4%2y2!+K^ia3/3!+...), . . (2) 

3 

and the c.g.f. about the mean of x is the same with the 
term in a removed. The second cumulant of x is clearly 
0{n}, and so the standard deviation is 0{n^). Let us 
therefore alter the scale so that xl\/n becomes the variate. 
The c.g.f. of this variate is then 




where is O(n^), /Cg is 0(1), is 0(n~*) and in general 
iCy is 0{n^-^^). Again, the c.g.f. about the mean is the 
same with the term in a removed. 

Thus as n increases the dominant term in the c.g.f. 
about the mean is which is the c.g.f. of a normal 

function 


<f>{z) 


1 






. ( 4 ) 


If, however, we retain the terms of smaller order, while 
choosing the scale so that /Cg — m.g.f. about the 

mean is 


M{a) = exp(ia2)exp(K3/f2”'a'®/3!+/c4/c2““a^/4! + ...) . 

== exp(|a2)(l+a3a3/3!+a4aV4! + ...) . . (5) 

where the second factor in brackets on the right arises 
from the expansion of the second exponential in the first 
line. Now if a probability function P{x), which vanishes 
with all its derivatives at the boundaries, has m.g.f. 

Jlf(o) = j’’p(x)e<^dx . . . (6) 

it may be proved by r integrations by parts that 

a-ilf(a) = (-r£^^yP(*)e“*da:. . (7) 


E 
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Thus here, reverting the m.g.f. of (6) term by term, we 
derive the corresponding probability function as 

p{z) = . (8) 

provided that the series for m.g.f. and probability function 
are convergent. This is the probability function of Type A. 

A close examination of the magnitude of terms in the 
expansion of 

exp(K3K^-ia^l3\+K^K^-^a*lil+...) . . (9) 

shows that the order of magnitude of coeflS.cients in the 
series of Type A is as follows : 

= 0(W’*), and = 0(n-^), ag, and ag = 

and later coeflSicients show a similar irregularity. 

Here let us pause to point out a practical disadvantage 
of the representation by Type A. If we are representing 
a given frequency distribution by Type A, we must use 
the observed moments to estimate the coefficients ag, ... 
in Type A. Let us suppose that the convergence demands 
the retention of terms up to 0(7r^), We must then include 
not only but also a^. Now depends on the 6th 
moment, and the 6th moment of the observations is subject 
to very high sampling error (68). Hence the effort to 
increase niatherriatical accuracy by retention of higher 
terms is largely frustrated by the statistical inaccuracy 
of the moments used to estimate those terms. 

Series of Type B. The procedure for deriving the 
function of Type B is rather similar. The f.m.g.f. proves 
to be 

exp(pa)(l +6gaV2!+63a3/3!+...), . . (10) 

which on reversion term by term gives 

p{x) = ^{a:)+6jjV*^(a:)/2I—63VV(=')/31+—. (H) 

the series of Type B, where denotes Poisson’s function 
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of 33 (3). Here the order of magnitude of coefficients is 
found to be : 

62 = O(n-i), 63 and 64 = 0 (n“ 2 ), 65 and 

and so on. Thus in using the function of Type B for the 
representation of a frequency distribution it is best to 
truncate the series after a difference of even order. 


36. Other Systems of Probability Fxmctions : the 
System of Pearson. We have seen how the functions of 
Types A and B arise by the addition of cumulant (or 
factorial cumulant) generating functions, corresponding 
to the compounding of values of an additive variate. 
But a variate of this kind is a very special one. For 
example, if x is built up of added increments, then 
which we might have occasion to use instead of x, is 
certainly not the sum of the squares of those increments. 
Indeed, as we may well anticipate, the distribution of x^ 
is different from that of x. 

For this and for other reasons the scope of t 3 rpical 
probability functions has been widened, and systems 
other than Type A and Type B have found acceptance. 
One such system is the system introduced in 1895 by 
Karl Pearson. 

Let us consider the difference or differential equations 
satisfied by some of the standard probability functions. 
We shall use the receding difference operation defined by 
S^<f,{x) ^ <j>{x)—<l>{x—l). 

(i) The binomial probability function of 22 ( 1 ) satisfies 


\7<f>{x) = 


a;—(n+l)p 
p(n—x-\-\) 


^(x). 


( 1 ) 


(ii) The Poissonian function ^(x) of 30 (4) satisfies 


V^(a;) = - ipix). 


• (2) 
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(iii) The hypergeometric probability function of 29 (2) 
satisfies 


V<l>{x) = - 


{M —a;+l)(w—x+l) 


(iv) The normal probability function in standard form 
31 (6) satisfies 

^ = -X<f>[x). . . . (4) 

A number of other probability functions, arising 
naturally in problems of repeated trials, might be added 
to this list. The Pearsonian system consists of the 
functions which satisfy the differential equation 

^ ( 5 ) 

dx Cq+CiX-\-C2X‘^ ’ ’ ^ 

The functions are found by immediate integration ; 
thus 

,_ r 


log y = — 


' Co+CiX+CaX*’ 


whence y can be found by the methods of elementary 
integral calculus. The quadratic in the denominator of 
the integrand may have real, variously positive or negative, 
or equal, or numerically equal but of opposite sign, or 
complex roots ; or again, with Cg == 0, may degenerate 
into a linear function, or with Cj and Cg = 0 into a constant. 
These various cases yield the Pearsonian curves, usually 
classified into twelve t 3 q)es ; while the discriminant of 
the quadratic, expressed in terms of moments of the 
curves, jdelds a “ criterion ” for judging in advance what 
type is appropriate to a proposed frequency distribution. 

A full account of the curves, their shape and the process 
of representing frequency data by them is given in 
Elderton^s Frequency Curves and Correlation (3rd edition, 
London, 1938), to which we refer the reader for details. 
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Here we have space to mention from time to time only 
a few of the curves, as they occur in special problems. 

37. Probability Functions Generated by Change 
of Variate. If x is distributed about the mean a; = 0 
in a normal distribution 

dp = {2'iT)-h^^^*dXy . . • (1) 

it is certainly not the case that x^ is normally distributed ; 
for putting z = ^x^, we have dx = {2z)~Hz, and so 

d,p = 7r~'h~h~^dz, . . • (2) 

The range of z is from 0 to oo, and the constant tt"* is 
such that the integral of the probability function of z 
over this range is 1. The distribution of z is skew, and 
is actually a case of Pearson’s Type III. 

Ex. 1. Prove that the m.g.f. of is (1 

Again, if x is distributed between — J and \ in the 
rectangular distribution dp = dx, the cube root 2 = a;* 
is distributed, as the reader should verify, in the U-shaped 
distribution dp = ^zHz, Or again, to take an example 
from physics, if the distribution of the velocities of a 
great number of particles about a zero mean velocity 
were normal, the distribution of their energies would be 
of Type III. 

The derivation of probability functions from the 
normal function by non-linear change of variate was 
emphasized by J. C. Kaptcyn in 1903 {Skew Curves in 
Biology and Statistics, Groningen), but was by no means 
a new conception even at that time. 

Ex. 2. If a; is a normal variato in standard measure, we 
have seen in Ex. 1 that the m.g.f. of z — is (1 —2a)~*. 
Hence the m.g.f. of H-irl + • • • 4-^^* where the are in¬ 
dependent normal variates with the same mean a: = 0 and 
in standard measure, is (1—2a)“i”. The probability function 
which has this m.g.f. is unique, and of the form 
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The reader should verify that this function actually has the 
above m.g.f., and should find by integration the value of c. 

Ex. 3. If X is distributed normally about a; = 0 as mean, 
find the distributions of: (i) « = e*, (ii) z = (iii) z = 


38. Cauchy’s Probability Function. The pro¬ 
bability function which we shall next consider arises by 
change of variate in a rectangular distribution. Let us 
take a point Q on the axis of y at unit distance from the 
origin O. Let a straight line be taken at angle d to QOy 
all values of 6 from — to being equally likely, to 
cut the X axis in the point X = (a;, 0). What is the 
probability distribution of a; ? 

The distribution of 6 is rectangular, dp = wW. 
Also X = tan 0, so that d — arctan a?, dO = da;/(l-[-a;2). 
Hence the distribution of x is given by 


, \ dx 


range —co to oo. 


The probability function appearing here is Cauchy’s 
probability function. It has the property (very awkward 
for any theory of estimation from sample based on 
moments) that its moments of even order /Ltg, ... are 
all infinite. The reader should verify this by integration. 
It follows at once that linear compoimding of independent 
variates obeying laws of Cauchy type cannot be carried 
out by the addition of cumulants ; in fact the cumulant 
g.f.’s do not converge. This exception to the common 
rule gives us a salutary reminder that linear compounding 
of independent variates does not necessarily generate a 
distribution of normal type. 

The Cauchy curve has been found to possess a specially 
remarkable property. If n independent variates obeying 
the same Cauchy law are added, and the mean is taken, 
this mean obeys exactly the same law. Not only so, but 
the distribution of any linear combination 


2 = CiX^+C^X^ + ,..+CnX, 
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of variates obeying the same Cauchy law, where the 
Cj are positive and sum to 1, is again exactly the same 
Cauchy distribution. 



The figure shows the normal curve in standard measure 
and the flatter Cauchy curve drawn to the same scale. 

39. The Pearson Curve of Type I. As a final 
example of a probability function arising from a particular 
problem, let us consider the following : 

Suppose that x is distributed in the rectangular 
distribution over the range 0 to 1. Let n+1 points 
be taken independently in this range. What is the 
probabihty that the point of these, as counted 

from the left of the range, is in the elementary interval 
x—\dx to x+\dx ? 

The probabihty is compound ; it is the probability 
that one, any one, of the n-{-\ points is in the interval, 
and that k of the remaining n are in the range 0 to x—\dx 
while n—fc are in the range x-\-\dx to 1. Hence the 
compound probabihty is 

dp = (l>{x)dx = W(fc)(w+l)a;*(l—. (1) 

for the first probabihty mentioned is (n-\-\)dx and the 
second is n^T^)X^{\—x)^~^, The probabihty function ^{x) 
obtained here is of Pearson’s Type I. It is in fact the 
integrand of the Beta function (Gillespie, Integration, p. 84), 
apart from the factor n(j.)(n+l) which ensures that the 
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area under the curve is 1. Had the range been a to 6, 
we should have obtained 

^(x) = (a;-o)»(6-a!)»-» . (2) 

The probability integral of the simpler form (1) over 
the partial range (0, x) is called the Incomplete Beta 
function. In the same way the integral over (0, x) of 
the function 

■ • • p" 

which is a case of Pearson’s Type III, is called the 
Incomplete Gamma function. 

Variety of Probability Curves. The preceding 
survey of types of probability function, though far from 
exhaustive, will have served to dispel the idea, once 
rather .prevalent, that normality and symmetry were the 
rule and that skewness was an accident of sampKng. The 
r61e of the normal distribution in statistics is not unlike 
that of the straight line in geometry ; and we do not 
force curves into the mould of the straight line. Skew 
distributions are in fact the predominant t 3 rpe, for skew¬ 
ness arises from Lexian variability or non-homogeneity, 
from Poissonian statistical rarity, from limitation in the 
number of causes of variation, and from non-linear 
transformations of the scale. 

Readers of fairly advanced mathematical attainments, and 
interested in the rigorous derivation of the normal distribution 
and its corrective terms, will find great profit in studying the 
Cambridge Tract No. 36, Random Variables and Probability 
Distributions (1937), by Harald Cram6r. 
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PRACTICAL CURVE-FITTING WITH 
STANDARD CURVES 

40. Representation of Frequency Data by Normal 
Curve. The present chapter will be devoted to the 
numerical details of representing frequency distributions 
by normal curves, curves of Type A, Poissonian curves 
and curves of Type B. 

In fitting the normal curve, that is, in finding the 
equation of the normal function of best approximation 
to the given frequency distribution, the idea is to represent 
the relative class frequencies by the corresponding segments 
of area under the normal curve between neighbouring 
ordinates corresponding to consecutive class boundaries. 
The mean m\ or m of the frequency distribution is taken 
as the estimate of the mean p, or of the normal function ; 
the second moment mg or s^y corrected for grouping if 
necessary by Sheppard’s correction, is taken as the 
estimate of the corresponding ox In order to use 
the standardized tables of the normal probability integral 
it is best, once m\ and Wg have been computed, to 
standardize the class boundaries, taking them as deviations 
from the mean, in units of s. The values of the probabihty 
integral corresponding to these class boundaries are then 
read from tables (Appendix 4) ; the first differences of these 
values are the estimates of the class probabilities ; and 
finally we may multiply by n, the total number in sample, 
to make comparison with the absolute class frequencies. 

Example. In the data of heights of Irishmen (18, Ex.) 
the mean is 67*34, and ma with Sheppard’s correction is 
4.705-0-083 = 4-622. Hence a = 2-16, I /5 = 0-465. The 
standardized deviations of class boundaries are shown in the 
column z = (a?—m±J)/5 below. Since their common differ- 
73 
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ence is 1/s or 0*465, they are readily found, when once any 
one of them has been computed, by repeated addition or 
subtraction of 0*465, and the results can be checked at the 
ends of the range. The next column shows the values of 
^ erf (z), the next the first differences of these, the next the 
same multiplied by 346, and the final column the original 
class frequencies themselves for comparison. 


X 

2 = (x—in±i)/8 

^ erf 2 

JA erf 2 

JnA erf z 

obs. 


— 00 

-0-5000 




59 

-3*646 

-0-4999 

0*0001 

0 

1 

60 

-3*181 

-0-4993 

0-0006 

0 

0 

61 

-2*716 

-0-4967 

0*0026 

1 

2 

62 

-2*251 

-0*4878 

0*0089 

3 

2 

63 

-1-786 

-0-4629 

0*0249 

9 

7 

64 

-1-321 

-0-4068 

0*0561 

19 

15 

65 

-0*856 

-0*3040 

0*1028 

36 

33 

66 

-0*391 

—0-1521 

0-1519 

53 

58 

67 

0*074 

0-0295 

0*1816 

63 

73 

68 

0*539 

0-2051 

0*1756 

61 

62 

69 

1*004 

0-3423 

0*1372 

47 

40 

70 

1-469 

0-4291 

0*0868 

30 

25 

71 

1-934 

0-4734 

0-0443 

15 

15 

72 

2-399 

0-4918 

0-0184 

6 

10 

73 



0-0082 

3 

3 


00 

0-5000 


— 






346 

346 
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41. Representation by Type A. The coefficients 
« 3 , « 4 , ••• in the series 35 ( 8 ) of Type A can be expressed 
in terms of the moments about the mean. For by 35 ( 5 ) 
the m.g.f. (in unstandardized scale) is given by 

1 +fjL2P.^I2\ +/X4a^/4! • . 

= exp(la^a^){l+aQa^a^l3\+a^a^a^l4:\-{-...). . ( 1 ) 

Multiply each of these expressions by exp(— 
and expand the product in the former case. Equating 
coefficients of a^/r I, we have the desired relations 

«3 = 

= (/^ 6 — 

~ (/^6 • • (^) 

and so on. 

The routine for fitting Type A is a slight extension 
of that used in fitting the normal curve. Moments about 
the mean are computed and if necessary corrected by 
Sheppard’s corrections. The coefficients a^, ... are 

estimated from these moments by the formulae just given, 
with substituted for /x,.. The integral of the corre¬ 
sponding Type A series is then taken instead of the normal 
probability integral. This involves the necessity, if terms 
in and are included, of having supplementary tables 
of the integrals of the functions which appear in these 
terms, that is, tables of 



and 



Such tables have been computed and are available. 
(British Association Tables, 1931 ; Bowley, Elements of 
Statisticsy p. 303, F^{z) only.) 

Example. (Bowley, Elements of Statistics, p. 309.) To 
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fit two terms of a series of Type A to data giving age 
distribution of St Louis school children in the sixth grade. 
(Age X means a; to a;+L) 

X 10 11 12 13 14 16 16 17 18 n 

nj 26 201 673 1001 739 310 80 13 1 3044 

By the usual routine we compute m'j = 13*665, mg = 1*498, 
mj = 0*366. Hence, using Sheppard’s corrections, the 
corrected 

= 1*498-0*083 = 1*416, 8 = 1*190, Ijs = 0*840, 


estimated Og = rn^/s^ = 0*211. 

The rest of the working can be arranged in columns as 



below. 







(1) 

(2) 

(3) 

(4) (5) 

(6) (7) 

(8) 

(9) 

(10) 

X 

t - 

1 erf« 

Ft{z) atFtii) 

(3)+(6) A 

nJ 

obs. 

normal 

10 

—00 

-0-6000 

-0-0666 -0-0140 

-0-5140 







0-0079 

24 

26 

38 

11 

-2*24 

-0-4876 

-0-0882 -0-0186 

-0-6061 






0-0678 

206 

201 

208 

12 

-1*40 

-0-4192 

-0-0904 -0-0191 

-0-4383 







0-2202 

670 

673 

630 

13 

-0*66 

-0-2123 

-0-0276 - 0-0058 

-0-2181 







0-3268 

996 

1001 

982 

14 

0*28 

0-1103 

-0-0076 -0-0016 

0-1087 







0-2440 

743 

739 

786 

15 

1*12 

0-3686 

-0-0766 -0*0169 

0-3627 







0-1024 

312 

310 

324 

16 

1*96 

0-4760 

-0-0942 -0-0199 

0-4651 







0-0264 

80 

80 

68 

17 

2*80 

0-4974 

-0 0756 -0-0169 

0-4815 







0-0042 

13 

13 

8 

18 

3*64 

0-4999 

-0-0672 - 0-0142 

0-4867 






0-0003 

1 

1 

0 

19 

oo 

0-5000 

-0-0565 -0-0140 

0-4860 


— 






3044 

3044 

3044 


The closeness to the observations is remarkable. Indeed 
the tests of “goodness of fit,” to be developed in 54, show 
that the discrepancies are so small as to be improbable, and 
the representation is unsatisfactory. We have here a case 
of “ over-fitting.” 

For comparison we have included in a final column the 
results given by the normal curve of best agreement. 


42. Representation by Poissonian Function or 
Type B. The coefl&cients b^, ... in the series 36 (11) 



FITTING A TYPE B CURVE 


77 


of Type B can be expressed in terms of the factorial 
moments /i, i^( 3 )» •••• ^'or by 35 (10) the f.m.g.f. 

is given by 

1 +fia +/X( 2 )a 2 / 2 ! +/i( 3 )a 2 / 3 !+... 

= exp()Lca)(l+62a2/2!+63a3/3!+...) . . (1) 

Multiply each of these expressions by exp(—jaa) and 
expand the product in the former case. Equating 
coefficients of a^/rl we have the desired relations 

^3 = /^(3)—V(2)/a+2/x2, 

^4 “ . . ( 2 ) 

and so on. Note that the numerical coefficients are the 
same as occur in 14 (5). 

The procedure of fitting by Type B is therefore to 
compute factorial moments of the data by the summation 
method (Appendix 2 ) and by substitution in the above 
formulae to estimate the coefficients 63 , 63 , ... of the 
Type B series. For the rest of the work we require the 
values of e~^m^lx\ and its differences of as many orders 
as may be necessary. 

The value of e“^ can be taken from a table (p. 147) of 
the exponential function. Then ne~‘^ is computed, after 
which each value of ne~'^m^jx\ can be obtained from 
the preceding value, corresponding to a;— 1 , by multiplying 
by mjx, most easily done by a calculating machine. The 
subsequent differencings and multiplication by coefficients 
feg and so on can best be followed from the illustrative 
example. 

Example. E. Rutherford and H. Geiger, in 2608 experi¬ 
ments {Phil. Mag., Ser. 6, 20, 1910, p. 69S) on the number x 
of a-particles radiated from a disc in 7'6 seconds, obtained 
the distribution : 

jcOl 23 4 6 6789 10 11 12-14 n 

nf 67 203 383 626 632 408 273 139 46 27 10 4 2 2608 
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The summation method for factorial moments gives 
m = 3'870, m(2) = 14-784, whence the estimate of 62/2! is 

*(14-784—3-87®) = -0-0965. 


The working is set out in columns as below. 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

X 

rnjt 

<3 

nV V 

inftjVV (2)+(6) 

obs. Poisson 

0 

64-40 

64-40 

64-40 

-6-25 

49 

57 

64 

1 

210-62 

156-12 

101-72 

-9-82 

201 

203 

211 

2 

407-37 

196-85 

40-73 

-3-93 

403 

383 

407 

3 

626-49 

118-12 

-78-73 

7-60 

533 

625 

526* 

4 

608-43 

-17-06 

-135-18 

13-05 

622 

632 

609* 

6 

393-52 

-114-91 

-97-85 

9-44 

403 

408 

394 

6 

263-81 

-139-71 

-24-80 

2-39 

256 

273 

264 

7 

140-34 

-113-47 

26-24 

-2-53 

138 

139 

140 

8 

67-89 

-72-45 

41-02 

-3-96 

64 

46 

68 

9 

29-18 

-38-71 

33-74 

-3-26 

26 

27 

29 

10 

11-29 

-17-89 

20-82 

-2-01 

9 

10 

11 

11 

3-96 

-7-33 

10-56 

-1-02 

3 

4 

4 

12 

1-28 

-2-68 

4-65 

-0-45 

1 

2 

1 

13 

0*39 

-0-89 

1-79 

-0-17 

0 

0 

0 


2608 2608 2608 

"N.B, —(i) In the dififerencings in columns (3) and (4) ^( —1), 
^(—2)... are tacitly taken as zero, (ii) The asterisked entries in 
column (8) have been raised from those in column (2) to make the 
totals of columns (7) and (8) both come to 2608. 

It will appear when we come to consider goodness of fit 
(54) that the representation by tho Poisson function alone, 
without the term in 6,, is satisfactory. 


43. Limitations on the Use of Moments in Fitting 
Curves. The discussion of the Cauchy distribution in 
38 has shown that moments are by no means always, or 
necessarily, the best parameters to use in representing an 
observed frequency distribution by a probability distribu¬ 
tion of assigned functional form. It depends entirely on 
the nature of the probability function what parameters 
may be used with adequacy. For example, since the 
mean of any number of observations x, each of which 
obeys the same Cauchy distribution, has exactly the same 
Cauchy distribution as x, it follows that the mean of sample 
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in this case is no more accurate, for the purpose of estimating 
the centroid of the curve, than any single observation; 
indeed it may be shown that the median is much superior 
for this purpose, while still better parameters can be 
found. Again, for the purpose of estimating the centre 
of an unknown rectangular probability distribution the 
mean of the n sample observations is quite a good 
estimate ; but surprisingly enough, as R. A. Fisher has 
shown, the mean of the two extreme observations alone is 
remarkably better. As a general precept it may be 
stated that for probability curves of shape and properties 
approximating to the normal curve the use of the mean 
and moments of the frequency distribution gives good 
estimates for those parameters in the probability distribu¬ 
tion ; but for other probability curves better parameters 
can be found. 

Example. Instructive material bearing on the rectangular 
distribution may be procured from Barlow’s Tables. The 
last two digits of the decimal parts of the cube roots of 
integers n, as given in Barlow, provide a distribution con¬ 
forming very closely to a rectangular distribution, of range 
00 to 99, and centre at 49*5. We may take the 60 or 49 
entries on each page, omitting the cases where n is a perfect 
cube, and we may record the mean of the last two digits 
over this whole sample, as well as the mean of the highest 
and lowest values only. The results, for each page from 
n = 100 to u — 999, will illustrate the accuracy of two 
different methods of estimating from sample the mean of a 
rectangular distribution. 
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PROBABILITY AND FREQUENCY IN 
TWO VARIATES 

44. Bivariate Distributions: Correlation and 

Regression. Hitherto we have been concerned 
exclusively with probability and frequency distributions 
in one variate, that is, with univariate distributions. But 
most of the important and interesting apphcations of 
statistics involve bivariate, trivariate or multivariate 
distributions. 

Let us consider how a typical bivariate frequency 
distribution may arise. Suppose that 1000 soldiers in a 
regiment are measured in height, x, and in weight, y. 
The measurements provide 1000 paired numbers (Xj^ y^), 
which may be plotted as points in a plane. The resulting 
assemblage of points may be called the “ dot diagram.** 



O X 


Now there may be, and in fact in the case of height 
and weight there is, a tendency for the value of to 
80 
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conform in some way to that of the corresponding Xj ; 
greater height as a rule is associated with greater weight. 
Any such tendency towards a functional relationship, 
obscured by random deviations, will manifest itself in 
the dot diagram by the greater density of the dots along 
a certain locus. This locus is not sharply outlined, but 
its estimation is important, for it is a smudged image 
of a curve which may be fine and clear-cut in the parent 
population of which the observations are a sample. This 
latent curve or functional relation y = F(x) is called a 
regreasiony the regression of y on x. It will be a matter of 
judgement what functional basis is chosen for its mathe¬ 
matical representation. Usually the representation is a 
linear one based on a set of prescribed functions Pi(r), 
P 2 (^)> •••> regression therefore appearing in the 

form 

y == <iro+aiPi(a:)+aaP2(^)+--M • • (1) 

to as many terms as are judged adequate. The statistical 
problem is then to determine the best estimates of Uj, 

... from the n paired observations y^). The 
functions Pi{x) are commonly polynomial or harmonic 
functions, but they may be of any preassigned functional 
type. 

The diagram of dots suggests a second point of 
view. The proportion of dots in an elementary region 
x—\ Ax< Ax, y—\ Ay < y < y+^ Ay gives an element 

of bivariate relative frequency which corresponds to a 
bivariate differential element of probability, let us say 
dp ~ y)dxdy, in the parent population. 

We may imagine that on each class-rectangle of the 
network of rectangles delimited by class boundaries of 
X and y a right prism is erected, of volume proportional 
to the corresponding class frequency. The tops of these 
prisms make a surface of flat terraces which we may call 
the prismogram, the analogue in three dimensions of the 
histogram. This prismogram, then, is the rough sampling 

F 
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approximation to an ideal probability surface z = <f){x, y), 
which is often called the correlation surface. 

The functional dependence of y on a; may be investigated 
either by the method of correlation, which consists in 
estimating the parameters of the bivariate probability 
function ^(x, y), or by the method of regression, which 
consists in estimating the coefficients in the regression 
function (1). Naturally the methods overlap to a certain 
extent. In the case of several important correlation 
(bivariate probability) functions the corresponding regres¬ 
sion curves are straight lines. 

45. Binomial and Hypergeometric Correlation. 

The natural extension of the twofold division, success and 
failure of an event which gives rise to the binomial or 
hypergeometric distribution in one variate, is an arrange¬ 
ment giving a twofold division in each of two events 
E and F, Such an arrangement is expressed by the 
fourfold tabhy as follows : 

Let the probabilities of the double events (E, F), {E, JP), 
{Sl, F) and (E, F) be p^^, p^, Poo- These are set 
out as shown in the fourfold table, the columns referring 


t T 



to E and the rows to F and F, The sum Pii+Pio» 
representing the total probability of E whether F occurs 
or not, is entered marginally as p ; and in the same way 
the other total probabilities p\ q' are entered margin¬ 
ally as sums of a row or of a column. 
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Contingency Table. The fourfold table is a simple 
example of a contingency table. The more general bivariate 
contingency table has h rows and h columns, corresponding 
to the division of one system into h categories and the 
other into k. If the probabilities are all rational 
fractions, it is possible to represent the bivariate population 
by a physical model, such as one of marked or coloured 
balls in due proportions. 

If E and F are independent events, then = pp\ 
Pio = n'y Pm. = ap' and Poo = ??'. so that = Pi^p^^. 

The determinant PxL'PQ(i~PiQPm fourfold table is 

thus zero. 

Ex. 1. Prove that this determinant is equal to Pn—pp' 
and to Poo~QQ'‘ 

Generating Functions. Just as in 7 we introduced 
a variable t to carry x as exponent in univariate generating 
functions, so it is natural to introduce u to carry y. The 
probability g.f. of a fourfold table will thus be 

0(t,U)==PiitU+Pi^+PgiU+PoO . . . (1) 

= 1)+^)h(<-1)(m-1). . (2) 

Ex. 2. Show that in the case of independence this splits 
into the two factors pt-\-q, p'u-\-q'. 

Now if we draw n times, with replacement each time, 
from the population characterized by the fourfold table, 
the g.f. will be 

• • • ( 3 ) 

The coefficient of t^u^ in the expansion of this g.f. will 
be the probability <f)(x, y) of having x cases E and y cases F 
in the n drawings. The function (f>{Xy y) is the correlation 
function of binomial type. 

Ex. 3. If the variates x and y are independent, show that 
y) is the simple product of the binomial probability 
functions 
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Again, if we have a fourfold population of N individuals 
with NpiQ, Npqi and Npqq individuals in the respective 
categories, and if we sample n times without replacement, 
the corresponding probability of x cases E and y cases F 
is the correlation function of hypergeometric type. Even 
when PuPoo ”" 2 ^ 10^01 = 0 an effect of correlation is induced 
by sampling without replacement. 

46. Bivariate Moments and Moment Generating 
Functions. The bivariate product moment of order r 
in X and ^ in ^ is defined by 

/i', = y)3fy» or <}>{x, y)x^y‘dxdy, . (1) 

XV J J 

or the corresponding mean values with Exdy or I dxE, 

xJ J y 

according to the discrete or continuous nature of the 
variables. 

There are three moments of the second order. If we 
take them with respect to the means and of the 
variates they are variance of x, p^^ the variance 

of y, and p^ the product moment of x and y, often called 
the covariance. 

Generating Functions. The bivariate generating 
function of probability is defined by 


Q(t^ u) = ZE<l)(x, y)t^u^ or j y)t^uHxdyy . (2) 

xy J J 

jjdi/ or jdxE. 


or the same with E 


Moment generating functions are defined by putting 
t = e^y u = ePy the general product moment p^^ being the 
coeflBlcient of a*‘j8*/r! si in the resulting m.g.f. 

Factorial moments can be defined by putting factorials 
and as defined in 29 (2), instead of powers x^ 
and y *; and a bivariate f.m.g.f. may be constructed by 
putting i =s 1 +a, u ^ 1 +j8. 
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Example. Prove that the f.m.g.f. of the fourfold table is 
(1 +pa)( 1 +p'P) +(PiiPa(, -PioPn)<^P- 


47. Normal Correlation as the Limit of Binomial 
Correlation. The g.f. of a sample of n drawings, with 
replacement each time, from the fourfold population is 

(p^tu +Pio^ +PoiW +Poo)” 

= [l+p(«~l)+p;(i.-l)+Pn(^-l)(^~l)r, . (1) 

by 45 (2). 

Just as at the corresponding stage in 31, let us consider 
the deviation of relative frequency of number of successes 
from means, rather than absolute frequency. We do this 
by putting t == u = eP in (1) and then writing a/n for 
a, j3/n for jS. We have then the bivariate m.g.f. 

[1 +pa/n +p' pin+p' 

= [(1 +paln+ip^a^lri^)(l +p'Pln+ip'^P^l7i^)(l +^{p-p^)a^ln' 

... ( 2 ) 


which tends asymptotically as n increases (the assumption 
throughout being that none of the probabilities in the 
fourfold table is 0(n~^)) to 

exp (pa+p'j3) exp ^(orfa^+2p(7ior2aj8+(T|j8^) . (3) 

where of == pqln, o| = p'q'In, pa^a^ = {Pii—pp')ln. 


Next, just as in the case of one variate treated in 31, 
and for analogous reasons, the question is to find a 
continuous function <f>(Xy y) satisfying 


00 

j* J if)(Xy y)e°^‘^Pydxdy 


= exp J((7fa2+2pcTiCT2aj3+(7|^2), (4) 


The answer provided by pure mathematics is that the 
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only function y) for which this is the case over a finite 
domain in a and j8 together is 

^{x, y) = --L==== exp j/)], 

27ro’i<T2 V1— 

where Q{x,y) = . (6) 

This function, the analogy of which with the normal 
probability function in one variable is evident, is the 
bivariate ruyrmal probability function or normal correlation 
function. The parameter p is called the coefficient of 
correlation. The reader will verify at once that when 
p = 0 the correlation function breaks up, as one might 
have expected, into the product of two ordinary normal 
functions, in x and y respectively. 

48. Properties of the Normal Correlation Function. 

Let us suppose that units of scale in x and y are standardized 
by putting cr^ = 1, Og = 1. The m.g.f. of the normal 
correlation function about the means then becomes 

exp i(a^+2paP+P^), . . . (1) 

and the coefficient of ajS/l! 1! in the expansion of this 
shows that p is the mean value of the product xy. This 
suggests that in computing the parameters of bivariate 
frequency distributions we should add to the usual four 
parameters of first and second order, namely the means 
w'oi variances and 5| of x and y, a fifth parameter, 
the mean value of the product of corresponding deviations 
X and y from the sample means. 

The standardized value of this mean product, namely, 

r = ^S(x!-m\Q){y-m^^)i8^8^, . . ( 2 ) 

corresponds in the sample to p in the population or 
probability function. We shall call r the Pearsonian 
coefficient, or producUmoment coefficient, of correlation of 
X and y in the frequency distribution. 
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liimits of r and p. The extreme values that r and 
p can take are 1 and —1. They cannot lie outside those 
limits. For, taking x and y as unstandardized deviations 
from their means, let us consider the mean value of 
{hx+hyy, or h^x^-\-2hkxy-\-k^y^y both in population and 
in sample, where h and k are arbitrary real numbers. In 
population the mean value is K^o\-\- 2 hkp(jy(j 2 -\-k^a\, in 
sample it is }i^s\-\- 2 hkTS^ 82 -\- 1 {?‘ 8 ^^, . . . • (3) 

Now these quadratic expressions in h and fc, being the 
mean values of squared functions, are of necessity not 
negative. But the necessary condition for this is that the 
discriminants 

should not be positive. Hence /)^<1, so that 

both p and r must lie in the range —1 to 1. 

The result, it may be noted, depends on a property of 
quadratic expressions, and therefore holds not merely for 
normal but for any distribution of x and y. 

Example. Prove that if x and y are uncorrelated and of 
unit variance, x cos d-{-y sin B and x sin B—y cos B are also un¬ 
correlated and of unit variance. 

In the case of independent variates, under any laws of 
distribution, the product moment about the means is 
zero. For if the separate m.g.f.’s of x and y about their 
means are 

then by compound probability the m.g.f. of the two 
together is 

(1+/^2o^V2^+•••)(!+/^o2^V2!+...), . (6) 

and since this has no term in aj8 we have = 0. 

It is most important to notice that the converse 
theorem is not true. The vanishing of does not imply 
independence. Consider for example the case when x is 
distributed in any symmetrical distribution about the 
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mean a; = 0, with variance a®. Then z = a;®—a* is also 
distributed about a zero mean. The variates x and z 
have complete functional dependence. Their product xz 
however is a;®—cr^a;, and the mean value of this is clearly 
zero. This is an extreme case, but it gives a sharp 
warning against inferring the existence of independence 
from a zero value of p, and still more from a zero value 
of r, which is merely an estimate of p. 

The normal correlation surface, when p = 0 and 
variances are standardized to unity, is a symmetrical 
bell-shaped surface which may be generated by the rotation 
of its central vertical section, a normal curve, about the 
vertical axis. When p 0 the surface acquires a hog¬ 
back ridge which lies in the first and third quadrants of 
(a;, y) if p is positive, in the second and fourth quadrants 
if p is negative. 

The loci of equal 'prohahility density (7) are found 
by equating <f}(x, y) to a constant, yielding curves of the 
form 

2/3a;y/ori<72+y*/°'i = c*. . . (7) 

These are homothetic ellipses. Among them the ellipse 
which includes a region in x and y of total probability J is 
sometimes called the “ probable ellipse,” a name, like 
“ probable error ” in 15, apt to mislead. This region is 
the bivariate analogue of the interquartile range (15). 

49. Regression Lines in Bivariate Normal Cor¬ 
relation. If we out the normal correlation surface by a 
series of planes all perpendicular to the axis of x, the sections 
are all normal curves. For each such section corresponds 
to a constant value of a;* of a:, and so the z-ordinate of such 
a section is, in standard scale, 

z = y) = c exp -p*)], . (1) 

= Ciexp[-J(y-pa:t)V(l-p% . . (2) 

where c and are constants; and this is the ordinate 
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of a normal curve with mean at y = and of variance 
1 or in unstandardized scale <71(1 ; this variance 

is the same for all such sections. The locus of the means 
of such sections is therefore the straight line y = px, or 
in unstandardized units yja^ = px/cxi. This straight line 
is the regression line of y on x. There is correspondingly 
a regression line x = py^ or xla^ = py/cxj, of x on y. 

The regression lines do not coincide unless p = ±1, 
in which case (with standard units) they are the bisectors 
of the angles between the x and y axes. If p = 0 the 
regression lines are the axes themselves; but the concept 
of regression is of little importance in this case. 

Note, The name ** regression ** was introduced by Sir Francis 
Galton (J. Anthrop, Inst., 15 (1886), p. 246). In bivariate 
data concerning heights of fathers, and heights of eldest 
sons, y^, he found that the regression lines, as estimated from 
the sample, were approximately y = ^x, x == ^y. This implies, 
for example, that if there is a group of fathers whose heights 
all deviate from mean height by d inches, then the average 
deviation of the height of their sons from mean height is only 
id. There is thus a tendency, in the next generation, to 
return or regress towards the mean. But there is no deep 
and remarkable significance in this ; it is a mere consequence 
of the fact that neither p nor r can numerically exceed 1, 
and in practice the values r -- -f 1 or —I are never found. 

50. Correlation Table : Computation of Product- 
Moment. A contingency table of h rows and k columns 
in which both variables x and y are metrical is called a 
correlation table. If x and y are continuous variates it 
will be convenient to take a class-unit of suitable size 
for each and thus to have class-frequencies corresponding 
to class-rectangles. For practical purposes it is advisable 
to choose these units so that each variate has ten or a 
dozen classes, not more. 

The following example illustrates the usual appearance 
of a correlation table. (The distribution is of Binet 
Intelligence Quotient, a?, and Verbal Score, y, of 500 Scottish 
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schoolgirls bom in 1921, tested in the first week of June 
1932. The Intelligence of Scottish Children^ Univ. of 
London Press, 1933, p. 96.) The score named 60 means 
60 and over, that is, the class 60 to 69, so that the class 
marked 60 in the report should be centred at 64'5; and 
so for other classes. The sums of rows and columns are 
entered in the margins ; they give the frequency distribu¬ 
tion of X when variation in y is neglected, and of y when 
variation in a; is neglected. 



X (Binet I.Q.) 

60 70 80 90 100 110 120 130 140 150 

/„ 

70 

2 

2 

60 

3 2 6 3 4 1 

19 

y 60 

10 15 26 19 14 2 

86 

(Verbal 40 

2 7 32 43 23 7 2 0 1 

117 

Score) 30 

2 28 50 31 15 2 1 

129 

20 

10 32 38 6 1 

87 

10 

11 28 4 

43 

0 

3 7 7 

17 

jr~ 

3 32 102 134 98 67 34 22 6 2 

500 


From the marginal distributions we can proceed to 
compute the means and mean-square-deviation &om 
means of x and y. This will always be the first step in 
computing r. The product-moment can be computed 
about provisional means, and then transferred by a 
correction to the true means, thus: 

Since ^ == wig, ^ ■2'y = »»oi> 

the product-moment about these means is 

= - . . . ( 1 ) 

Hence, just as mean-square-deviations can be com- 
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puted about a provisional mean and transferred (14) to 
the true mean by subtracting a square, so mean-product- 
deviations can be so transferred by subtracting the 
corresponding product of deviations of provisional means. 
It is to be observed that this product may be negative, 
in which case the correction involves an addition. 

Several different methods of computation are in use 
for finding r. We shall exemplify two, of which the rest 
are mostly variants. 

(i) The first method consists in computing SExy 
piecemeal according to the contributions made to this 
sum by the frequencies in the rows, or alternatively in 
the columns. For example, in the row, for y 
constant, we compute 2 that is, multiply each class- 
i 

frequency by the value of x, Xf, and add along the 
row. For the different rows we may enter these values 
Ef^Xf in a suitable column to the right. The sums of 

such values for all rows gives Ex, and so may be used 
to check the mean ; while if we multiply each entry 
in that added column by its appropriate and sum down 
the column we have EExy, 

The same procedure may be carried out by colunms 
instead of rows. We then have a check on both means 
and on EExy, The whole scheme can be neatly arranged 
in rows and columns annexed to the table as below. The 
special value of the arrangement is perceived when it is 
found necessary to compute correlation ratios (52) as well 
as correlation coefficients. It simplifies the arithmetic, 
too, to choose units such that the class-breadths of x 
and y are both unity. 

Ex. 1. By way of explanation of the entries, note 
that the second entry, 63, in the Ex column comes from 
3x1+2x2+6x3-1-3x44-4x5+1x6, while the second 
entry, —51, in the Ey row comes from 

2xl+2x0+10x(-l) + llX(-2)+7x(-3). 
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X 



-8 

-2 

-1 

0 1 

2 

3 

4 

6 

0 

4 

V vJ 

V*/ 

Zx yZx 

4 







2 



2 

4 8 

32 

8 32 

3 




3 

2 

0 

3 

4 

1 

19 

3 67 

171 

63 189 

2 




10 15 

26 

19 

14 

2 


86 

2 172 

344 

190 380 

V 1 


2 

7 

32 43 

23 

7 

2 

0 

1 

117 

1 117 

117 

113 113 

0 


2 

28 

60 31 

15 

2 

1 



129 

0 0 

0 

39 0 

-1 


10 

32 

38 0 

1 





87 

-1 -87 

87 

-44 44 

-2 


11 

28 

4 






43 

-2 -86 

172 

-60 100 

-3 

8 

7 

7 







17 

-3 -61 

163 

-30 90 

/* 

3 

32 

102 134 98 

67 

34 

22 

0 

2 

500 

130 1070 

289 048 

X 

-3 

-2 

-1 

0 1 

2 

3 

4 

5 

6 







-04 

-102 

0 98 

134 102 

88 

30 12 

289 





27 

128 

102 

0 98 268 306 362 150 72 

1603 

*"io 

289/600 - 0-678. 

ZV 

-9 

-61 

-102 

6 76 

80 

63 

47 

16 

4 

130 

moi - 

130/600 - 0-260. 

x'^y 

27 

102 

102 

0 70 160 189 

188 

80 24 

948 


1603/600 


-(0-678)« « 2-072. 


«2 “ 1076/500-(0-260)* «= 2 084. 
- 948/600 ~(0-578)(0*260) - 1-740. 

Hence 

= 1*636, = 1*444, r = 1-746/1 635X 1*444 =+0*74. 


Error of Sampling.* It is desirable here to anticipate 
Chapter VII a little. The value of r is the estimate of p 
obtained from one sample of n bivariate measures. Now 
each possible sample will yield its own value of r, and the 
ensemble of such values constitutes the sampling dis¬ 
tribution of r. This, when the parent population is normal, 
is a skew distribution, first studied (78) by R. A. Fisher. 
As n increases it tends slowly to the normal type, with 
mean p and variance (\—p^)^j{n—\). Thus the standard 
error (68) of r is approximately (l—r^)ly/n\ but only 
when n is large, say ?i>100, and |p| is not too great, 
say |p|<0*5, can we use normal theory. In most cases 
it is better not to use standard error, but to proceed as 
in 78. 

(ii) The second method of computing r depends on 
the simple observation that while by summing the 
frequencies in columns we obtain the distribution in x 
alone, and by rows that in y alone, if we sum along 
diagonals inclined at 46® to the horizontal we obtain a 

* This paragraph may be postponed until Chapter VII has 
been studied. 
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distribution of x—y ; for all class-rectangles in any such 
diagonal correspond to the same value of x—y. Thus 
from diagonal frequencies we may compute the mean of 
x—y^ namely rw-oi, thereby checking the individual 
means as computed from row and column marginal 
frequencies ; and we may also compute the mean-square- 
deviation of x—y from its mean. Now the value of this is 

But and are already known from the row and 
column marginal distributions ; hence r is easily found. 

Ex. 2. Taking the same example as before and summing 
along the diagonals, we find the frequency distribution of 
x—y to be 

x-y -3 _2 -1 0 1 2 3 4 5 JV 

Nf 2 22 87 173 149 57 8 1 1 500 

The mean is found to be 0*318, checking the values 

= 0*578, = 0*260. The mean-square-deviation from 

the mean is 

52 —2rs^52-f5| — 1*265, 

whence 

r - i(2*672+2*084-l*205)/l*635x1*444 
=-- ix3*491/2*361 = 0*74, 

as before. 

Notice that we have here no check on r. That could be 
provided by summing along the other set of diagonals at 
right angles to those which have been taken. They corresi)ond 
to constant values of x+y, and so their moan-square- deviation 
from the mean is 

Ex. 3. The distribution of x-\-y, obtained by summing 
along the other set of diagonals in the correlation table, is : 

x+y -6-6-4-3 -2 -1 0 1 2 3 4 5 6 7 8 9 N 
Nf 3 7 18 38 38 68 63 64 68 40 37 23 20 6 6 1 600 

Compute r from this distribution. Notice how much more 
widely spread it is than that of x—y in Ex. 2. 



94 PROBABILITY AND FREQUENCY IN TWO VARIATES 


Sheppard’s Corrections. Sheppard’s correction for 
variance in grouped data is applicable to the mean-square- 
deviations of X and y, but not to the mean-product- 
deviation. On the whole, however, it is better to work 
without the corrections, because the tables of Fisher’s 
sampling distribution of r do not take account of grouping. 

61. Correlation of Variates with Poissonian Dis¬ 
tribution. It is not necessarily true that sampling from 
a fourfold population always produces as a limiting case 
a bivariate normal correlation function. Suppose, for 
example, that p and p' are of order 1/n. Then may 
be of order 1/n^, but may also be of order 1/n. 

The f.m.g.f. of a sample of n individuals with replace¬ 
ment is seen from 47 (1) to be 

(l+pa+p'p+Pnap)\ . . . ( 1 ) 

When p = fijn and p' == pfjn, and p^ is 0{lln^), this 
g.f. tends to exp {fia+fx'p), which shows that with in¬ 
creasing n the probability function reduces to the product 
of independent Poissonian functions, and is in fact 

y) = e"/* • • * (2) 

On the other hand, when p^ is we have 

(1 +pa+p'^+Piia^)’* 

which tends to 

exp . . (3) 

where 

ii = MPii-PP') == MPnPoo-PioPoi)- • • W 

Evidently fi is the ordinary product-moment about the 
means. 

Now putting a = f—1, j8 = w~l in (3), we derive the 
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correlation function ^(a;, y) as the coefficient of t^vy in the 
probability g.f. 


It is found without difficulty to be (in one of its forms) 

' xt u l 


X 1 + 




x\ y\ 
pyx{x - 


iMy-i) 


(/*—A)(m'“A) (/*——p)*2! 


...] 


( 6 ) 


where the polynomial in the bracket terminates after x+1 
or ^ + 1 terms, whichever is the lesser. This function is 
the bivariate Poissonian function. It may be proved that 
the loci of means of sections corresponding to constant 
X ov y are straight lines, so that here again we have linear 
regression. The same property may be proved to hold 
for binomial and hypergeometric correlation functions. 

Both the normal function and the Poissonian correlation 
function can be derived, like the corresponding functions 
for one variate, on more general grounds than sampling 
from a fourfold table, by a compounding of elementary 
increments achieved by addition of bivariate cumulant 
g.f/s ; but this derivation lies beyond our present scope. 


52. Non-Linear Correlation and Regression. A 

linear regression between correlated variates is rather 
exceptional. The loci of means of arrays usually deviate 
from straightness by more than can be ascribed to random 
sampling, suggesting that the underlying law of probability 
cannot be either normal or Poissonian. Non - linear 
regression curves are perhaps best estimated by fitting 
to the data suitable regression functions by the method 
of Least Squares, described in Chapter VI. In the 
non-linear case, too, the coefficient r or p has marked 
disadvantages (it was seen for example in 48 that p could 
be zero even when regression was perfect) and the cor¬ 
relation ratio 7^, devised by K. Pearson, is much to be 
preferred. 
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It was proved in 49 that in normal regression all 
^•sections or arrays corresponding to constant x had the 
same variance, let us say 

a2i = a|{l-p2), . . . (1) 

80 that 

l-P* = <^1.1/4 • ■ • (2) 

Now may be regarded as measuring something 

complementary or antithetic to correlation. The word 
alienation is sometimes used to describe this quality, but 
alienation suggests repulsion and is too strong a term. 
Residual dispersion expresses the meaning better. In 
non-linear regression the variance of the 2 /-sections, namely 

^(y-9rV<f>(x,y)dy^^^(x,y)dy, . . (3) 

where 

jy^(.^>y)dy I j4>i^,y)dy, . . . (4) 

the mean of the y-section corresponding to constant x, 
is not usually constant. We may, however, take the mean 
of these variances of i/-sections over all sections, that is, 
over all values of x, obtaining 

j j{y-yg)^<l>(z,y)dxdy, . . (6) 

which may be regarded as the mean-square-deviation of 
y from its regression value y®, taken over the whole 
distribution. Standardizing this by dividing by the total 
variance of y, namely a|, and writing 

i-4x = 4i/4 • • • (®) 

we define a coefficient rjy^ analogous to p in (2). This 
coefficient is the correlation-ratio of y on x. The closer 
it approaches 1, the smaller is the residual dispersion and 
the closer the values y lie to their regressional means. 
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In the same way, by interchanging the rdles of x and y 
in the above derivation, we define the correlation- 
ratio of a; on ^. As to the signs of and r^^y, there are 
cases where these can be attributed by graphical or other 
considerations, but there are also cases, for example when 
the curve of regression is a periodic curve with several 
oscillations, when sign has no meaning. 

The estimates of riy^ and as derived from an actual 
frequency distribution presented as a correlation table, 
will be denoted by Cy^ and e^y. We define them 
analogously ; thus 

• • • ( 7 ) 

where s\ ^ is the mean, over all ^/-arrays (columns of the 
correlation table), of the mean-square-deviation of y from 
the mean y^ of the colunm. In computing this mean of 
mean-square-deviations the column frequencies, marginally 
entered, serve as class frequencies. The effective arith¬ 
metical arrangement of the computation will be given later. 

That the correlation-ratio is actually a ratio, namely 
the ratio of the standard deviation of the means of arrays 
to the total standard deviation of the variate, will now 
be proved by considering By^,, 

Lemma, If k sots of n-a, observations, 

with respective means Mj and mean - square - deviations 
j — 2, k, are pooled in an aggregate of 

n — + observations with mean M and mean- 

square-deviation 5®, then 

na^ = . . . (8) 

i ^ ^ 

where Cf = M —Mf. 

This follows at once from the fact that the mean- 
squarc-doviatioii of the j** set about M is 

Applying this lemma to the column-arrays of a 
correlation table, we have 

nsl = 2ni(s^^+Mj), . . . (9) 


o 



98 PROBABILITY AND FREQUENCY IN TWO VARIATES 


where and are the mean and mean-square-deviation 
of the column. (The origin is the mean of both x 
and y,) This is the same as 

. . . ( 10 ) 

the second term denoting the mean-square-deviation of 
column means, when these are associated with column 
frequencies. 

The above result holds for the sample. A similar 
result can be obtained for the population, integrals 
replacing sums, and variance replacing mean-square- 
deviation from means. The result may be put in words 
thus : total variance of y is equal to mean of variances 
^ 2 ,x y-arrays plus variance of means of arrays. It 
is another example of analysis of variance (26, 76), 

Hence, by (6) and (7), 

"nix = ^uj°l • • ( 11 ) 

so that and are displayed as ratios of variances 
or of mean-square-deviations. 

53. Computation of Correlation-Ratios. The result 
52 (11) permits us to compute and by a simple 
extension of the first method of 50 for computing r, for 
the means of rows and columns are given by the entries 
in the column headed Zx and the row headed Zy, divided 
respectively by the frequencies fy and /». Also, the means 
of these entries are and Hence, computing mean- 
square-deviations from means in the usual way, we have 

= [^^(■2'y)®//«-m;2]/s|, . . (1) 

and similarly for e^. We thus annex two rows 
(Zy) ^Ifx, and two columns (Zx) (Zx) ^fy, to the computation 
scheme for r. 
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Example. The additional rows and columns for the 
example of 60 (Binet I.Q. and Verbal Test Score) are as 
follows : 


(^y)' 81 2601 10404 36 5776 6400 3969 2209 256 16 I 

(Z?/)*//® 27 81 102 0 69 96 117 100 43 8 633 

- [633/500-(0-260>»]/(l*444)‘ 

= 1-198/2 084 - 0*575 

«L = [916/500-(0*578)»]/(l*636)* 
xy 

« 1*496/2.672 - 0*560. 


(Ex)' (ExflSy 

64 32 

3969 209 

36100 420 

12769 109 

1521 12 

1936 22 

2500 58 

900 53 

915 


Hence ey^ and are equal to 0*76 and 0*76, whereas the 
value of r was found to be 0*74. 


64. Correlation of Non-Metrical Characters. When 
the characters in a double classification are purely quali¬ 
tative, capable of being graded by a recognizable difference 
in category, but not susceptible of measurement by metrical 
scale, we must fall back on the contingency table oihxk 
rectangular cells, with corresponding cell-frequencies. 
Since variances and product-moments are now out of the 
question, the presence or absence of correlation must be 
inferred from the ceU-frequencies themselves, according 
to the manner in which they deviate from presumptive 
cell-frequencies in the corresponding case of independence. 

CJonsider, for example, the following contingency table 
due to Galton (Proc. Roy. Soc., 40 (1886), p. 42), illustrating 
the incidence of eye-colour in a group of fathers and eldest 
sons. 



Pi 

El 

El 

E, 

p’ 


0-194 

0-083 

0-026 

0-056 

0-368 

(i) P, 

0-070 

0-124 

0-034 

0-036 

0-264 

P. 

0-041 

0-041 

0-066 

0-043 

0-180 

P, 

0-030 

0-036 

0-023 

0-109 

0-198 

P 

0-336 

0-284 

0-137 

0-244 

1-000 
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El E^ E^ E^ j p' 

E^ 

(ii) F, 

F, 

F, 

P 

The colour c.itogories are 1, blue; 2, blue-green or grey; 
3, dark grey or hazel; 4, brown, n = 1000. 

Summing down columns we obtain frequency estimates of 
the probabilities p of respective eye-colours for fathers 
irrespective of sons, and summing along rows, frequency 
estimates of probabilities p' for sons alone. These marginal 
frequencies or relative frequencies may be recombined 
again to form a multiplication table, which is to serve 
for comparison with the original table. The marginal 
frequencies in the second table are the same as in the first, 
but the cell-frequencies, derived as they are by applying 
the law of compound probability, represent what would 
have been the state of affairs with the same marginal 
frequencies had there been independence. Of course it 
must be observed that if we use, as here, not the a priori 
marginal probabilities but only the sample estimates given 
by the marginal frequencies, this procedure is bound to 
affect the sampling probability of the coefficient or criterion 
of comparison, 

The coefficient is a quadratic function of the 
deviations of cell-frequencies in the actual from those 
in the presumptive independent case ; it is a kind of 
composite weighted variance, with application not merely 
to contingency tables but also to any comparison of actual 
frequency classifications, single or multiple, with pre¬ 
sumptive ones. It was first employed by Lexis, but the 
nature of its probability distribution was first obtained by 
K. Pearson in 1900. 


0*120 

0*089 

0*060 

0*066 


0*102 
0*075 
0 051 
0 056 


0*049 

0*036 

0*025 

0*027 


0*087 

0*064 

0*044 

0*049 


0*358 

0*264 

0*180 

0*198 



0*335 0*284 0*137 0*244 1*000 
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Distribution of The derivation of the 

distribution involves the general multivariate normal 
correlation function, which is outside the scope of this 
short book ; but the outlines may be sketched. If the 
a priori probabilities in the k classes are pj, p 2 » •••» Vky 
then the frequencies in the classes are characterized by 
the multivariate multinomial g.f. 

(Pik+P4i+—+Pkh)'‘- • • . ( 1 ) 

If the number of individuals found in a class is 
the expected number being np^y we may denote the class 
deviation from mean value or expectation n^—npj by 
Then, since = n and also Enp^ = n, we must have 

i:€, = 0, . . . (2) 

a relation in virtue of which only Aj—1 of the deviations 
let us say the first k—\y are indcyjendent. We therefore 
put 4=1 consider what happens as n 

increases. Putting tj = e®#, we find that, provided no p^ 
is 0(n~^), the multivariate m.g.f. of the class deviations 
tends to 

exp [ire 2{pjqja^-2pfPiafi))] . . (3) 

t, j « 1 

This is an m.g.f. of normal correlated probability in 
the k—l deviations, which on reversion gives the probability 
differential of the as 

k 

c exp [—ire-’ref/p^lffeirfcg ••• *i-i- ■ • (4) 

Thus the probability, or probability density, of a set 
of deviations is a function of the quadratic expression 

X^==Se?lnp„ . . . (5) 

which is Pearson’s Having decided to use the com¬ 
posite rather than the individual deviations as a 
criterion of the nearness to expectation, we transform the 
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differential (4) into a differential in itself, when it 
assumes the shape 

dp == • • • (6) 

the probability function being of Pearson’s Type III or 
Gamma type. 

The probability of obtaining a value of not exceeding 
a given Xo is therefore 

Pxl- f f . (7) 

Jo Jo 

Tables of this function P have been computed for various 
values of k, the number of classes, and x^- 

Degrees of Freedom in x^- When the class pro¬ 
babilities Pij are given d priori the distribution of x^ ^or 
k classes is expressed, as we have seen, by 

dp = . . . (1) 

But the presumptive class probabilities are not always 
given a priori ; in a contingency table, for example, they 
may be estimated by recombining in multiplication the 
marginal relative frequencies of the table which is being 
tested. Now such a procedure forces the marginal totals 
of the presumptive table of independence to agree with 
those of the contingency table. This forcing reduces the 
number of independent class deviations from expectation. 
For example, in a 4-by-6 table there are 24 cleisses, of 
which 23 have independent frequencies, since the total of 
relative frequencies must be 1. This is in the absence 
of forcing. On the other hand, if the 10 marginal totals 
are preassigned, then there are only 3x5 or 15 inde¬ 
pendent clsuss frequencies, as may be seen by putting these 
15 in the top left part of the table, so as to fill 3 rows 
and 5 columns, and observing that all the others can then 
be filled in by reference to the marginal frequencies. 
In general, in an hxj table with forced maiginal 
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agreement, there are only (^—1)(J—1) independent class 
frequencies. 

Now, in preparing for comparison by the ;^2.test in such 
a case, we should not integrate c exp ••• ^*-1 

over all the previously independent for by so doing we 
would be unfairly including combinations of the which 
have been precluded by the procedure of forced agreement. 
We ought to transform so that it is expressed in terms 
of the restricted set of independent It was shown by 
R. A. Fisher that when this is done the modified element 
of probability is simply 

dp = * . (2) 

where m is the number of restrictive relations, reducing 
the number of independent from A;—I to k—m—l. It 
is usual to call Jc—m—l the number of degrees of freedom. 

The table of P{x^) is therefore best constructed, and 
consulted, with reference not to k, the number of classes, 
but to k—m—ly the number of degrees of freedom ; and 
this applies not only to contingency tables but to all 
situations in which a presumptive probability distribution 
is obtained from a frequency distribution by a partial 
forcing of agreement, the equating of moments for example, 
involving restrictions on the deviations These restric¬ 
tions must be linear, that is to say, they must involve the 
in the 1st degree only. 

Since in the deduction of P(x^) we excluded the case 
of very small class probabilities, we must exclude in 
practice small class frequencies. It is customary, there¬ 
fore, in applying the test, to pool the small frequencies 
at the ends of a distribution so as to make the classes 
contain at least 10 individuals. 

Example. The fitting of Poissonian and Type B functions 
to the Ruthorford-Goiger data in 42. We pool the classes 
corresponding to a? = 10 and over. Thus A? = 11. 

For the Poissonian fitting there are 9 degrees of freedom, 
since the total frequency and the mean have been made to 
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agree in fitted curve and data. We find ^ — 12* 8, and 
reference to tables shows that P = 0*20, a satisfactory value. 

For the Typo B there are 8 degrees of freedom, total 
frequency, mean and second factorial moment having been 
made to agree in fitted curve and data. We find ^ — 10*2, 
P == 0*25. The slight improvement is of little consequence ; 
in both cases the principal contribution to ^ comes from 
the large deviation in class a? = 8. 

Empirical Formula. The value of for which 
P = 0-05 is often regarded as a boundary between the 
reasonable and the dubious. This value of ^ is given 
with adequate approximation, for y degrees of freedom, by 
l*55(A:'+2), and l*25(ib'+5), y>l0. 

For k' — 35 the second formula above gives the value 50, 
the actual value of b^ing 49*80. For higher values of k', 
-y/2x*“” V 2k' may be treated as a standard normal variate. 

55. Goefiicients of Contingency. The possibility of 
dependence between variates in a contingency table can 
be tested by P(x^)- For Galton’s data of eye-colours in 54 
the value of 2 is 266, a value so large that the probability 
of independence of eye-colour between fathers and eldest 
sons is negligibly small. 

Attempts have been made to measure the strength of a 
dependence by means of coefficients of contingency. Thus 

measures, as it were, the dispersion of a grouped sample 
from expectation, taken over all n individuals ; and so 
the mean dispersion per individual is ^ coefficient 

denoted by and called by K. Pearson the mean square 
contingency. Since 

^ 2 = ;f«/M = 2'(e,/n)2/p„ . (1) 

it apy^ears that ^ is the sum of squared deviations of class 
relative frequencies e^jn from the presumptive class 
probabilities each divided by that probability p^. 

Pearson, considering the value of for a bivariate 
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normal correlated distribution divided into grades of 
indefinite fineness in x and y, found the relation 

p^ = <f>^l(l+,t>% . ( 2 ) 

and, proceeding by analogy, defined a general coefficient of 
mean sqvxire contingency C by 

= . . . (3) 

Evidently is zero when (f)^ is zero and tends to 1 as 
increases ; but its interpretation for intermediate values is 
not very definite. 

Example. The computation of and C* for Gallon’s 
data in 64. 

The table of values (pn—PiP'^)^IPiP^ is : 



E, 

Fz 

Fz 

F, 


Fx 

0-046 

0-004 

0-012 

0-011 

0-073 

F2 

0-004 

0-032 

0-000 

0-012 ! 

0-048 

Fz 

0-006 

0-002 

0-036 

0-000 

0-044 

F, j 

0-020 

0-007 

0-001 

0-073 

0-101 


0-076 

0-045 

0-049 

0-096 

0-266 


Thus = 0-266, = 0-266/1-266 = 0-210, C == 0*46. 

Table of P(x^)- A table of P(x^), arranged in a 
compact and practical form, is given in Table III of 
R. A. Fisher’s Statistical Methods for Research Workers^ 
8 th edition, pp. 110-111 ; also in the Statistical Tables for 
Biological, Agricultural and Medical Research of Fisher and 
Yates (Oliver and Boyd, 1943), p. 27. 

For practice in the tho reader may examine 

whoth('r tho experimental dat<i of tho examples on pp. 49 
and 60 are in good accord with the theoretical distributions, 
rectangular and binomial, there given. 
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THE METHOD OF LEAST SQUARES: MULTIVARIATE 
CORRELATION : POLYNOMIAL AND HARMONIC 
REGRESSION 

56. Multivariate Regression. When distributions in 
more than two correlated variates are encountered, an 
important question is the determination of the optimal 
value (sometimes in the sense of mean value, sometimes 
in the sense of most probable value) of a particular variate 
in terms of the values of all or any given set of the other 
variates. We have seen that in normal bivariate dis¬ 
tributions the loci of such optimal values are straight 
regression lines. In normal correlation of many variates 
the corresponding loci are still linear, expressed by 
equations of the first degree. For three variates there 
are three planes of regression, for n variates there is a 
sheaf of n hyperplanes, each given by a linear equation 
expressing a particular variate in terms of the other n—l 
variates. 

It was proved by Yule that these various linear loci 
could be obtained without the assumption of normal 
distribution by using the method of Least Squares, which 
we now describe. 

67. The Method of Least Squares. The method of 
Least Squares originated in the practical necessity of 
combining discrepant observations of a single unknown 
constant, or discrepant observational equations in several 
unknowns, in such a way as to obtain best estimates of 
the unknown or unknowns, imder some accepted criterion. 

100 
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Discrepant measures are inevitable in repeated obser¬ 
vations, even when every effort has been made to keep 
conditions constant. The conditions can never be identi¬ 
cally realized a second time. However delicate the 
instrument of measurement, there are innumerable fine 
and uncontrollable variations inherent in its parts and 
their adjustment and the readings, to say nothing of the 
inaccuracies of the observer. Hence, just as in the 
throwings (4) of a coin, we have varying phases of a 
system S. Thus repeated measures of a supposedly 
unique physical constant are foimd to be discordant, 
the truth being that they are a sample from a certain 
probability distribution depending on S, In the same 
way, when linear combinations or other functions of 
several unknowns are measured, the number of observations 
exceeding the number of unknowns, the equations so 
derived are nearly always found to be inconsistent. 

In 1805 Legendre proposed, as a convenient method 
for reducing certain astronomical observations, that the 
“ best value should be taken as that for which the sum 
of squared deviations of the observations was least. This 
is the principle of Least Squares, It can be justified under 
the assumptions (i) that the measures are normally 
distributed and (ii) that the best value has maximum 
probability density. This derivation is mathematically the 
simplest and most rapid, but it unduly limits the types 
of error distribution. A more comprehensive derivation 
postulates that the best value is (i) a consistent or unbiassed 
linear combination of the observations and (ii) has minimum 
variance. It is remarkable that the two quite different 
sets of postulates lead to exactly the same equations for 
the unknown or unknowns. 

58. Precision, Weight, Errors and Residuals. 

Measuring instruments of differing precision may be 
characterized by their standard error, or variance of error, 
in the reading given by them of some assigned measure. 
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The variance may be estimated by repeated trials. It is 
traditional here to use the term weight, defined as propor¬ 
tional to the reciprocal of variance of error. For example, 
if in determining a distance of 5000 yards the standard 
error of a range-finder A is estimated to be half that of 
a range-finder B, the weights w^ and assigned to 
readings made by A and B would be as 4 to 1, in favour of A . 

Finally, it must always be kept in mind that “ true ” 
values (if indeed the word “ true ” admits at all of definite 
meaning) are unknown and must remain unknown; so 
that the errors, being deviations from an unknown value, 
are likewise unknown. True values must be estimated by 
appropriate substitutes, namely, best or optimal values, 
and errors by the deviations of the observed from the 
optimal values. These deviations are distinguished from 
the errors which they represent by being called residuals. 
Errors are €/ = a^—a, residuals are = a^—d, where a 
is the true value, an observed value of a, and d the 
optimal value of a. If there are n observations, the n 
residuals are estimates of the n errors ; and the n errors 
are themselves only a finite selection under the law of 
probability which characterizes the circumstances of 
measurement. 

59. Repeated Measurements of a Single Unknown. 

The estimate by Least Squares is found by minimizing 
the sum of weighted squares of residuals. The minimum of 

— Ewj{x^—x)^ . . . (1) 

i 

is given by dS^/d^ = 0, so that 

A = EwjXjjEwf. . , • ( 2 ) 

The optimal value of x thus appears as a weighted mean 
of the observations. If the observations are all of 
equal weight the optimal value is thus the arithmetic 
mean. 
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Variance of Optimal Value. The variance of i in 
(2) is (15) 

UwjajKZwjf = (Swp‘^jWj)/(SwjY = o^jEwi, . (3) 

where a? is the variance of Xj and is the variance of 
an observation of unit weight. Thus the weight of x is 
Ew^^ the sum of the weights of the x^ In particular 
the weight of the arithmetic mean of n values x^ all of equal 
weight, is n times the weight of any x^. 

Variance of Residuals in Case of Equal Weight. 
If the observations are all of unit weight the residual 
is 

Xi—x==^ (n-\)Xiln-~(x^+x^+,..-{-x^~Xi)jn. . (4) 
Thus the variance of is (15) 

(n—1) V7w2+(n--l)<7Vn^ = {n—l)a^ln. . (5) 

It follows that an estimate of cr^ is given by dividing 
the sum of squared residuals not by n but by n—1. 

Ex. 1. The author made 30 bisections by eye of lines of 
constant length. The distribution of a?, the length in cm. of 
the segment to the left of tlie point of bisection, was ; 

X 7-6 7-66 7-75 7-8 7-85 7-9 8-0 8-1 8-15 8-2 8-26 8-45 n 
n/ 23 1 44242322 1 30 

Estimate the length of the half line and the standard error. 

Ex. 2. Do the samo for the results given by a second 
person : 

X 7*7 7-75 7-8 7*85 7*9 7-95 8-0 8-05 8-1 8-16 8-2 8-3 n 
n/ 1 1 1 4 3 5 4 5 3 1 1 1 30 

Ex. 3. Compare the precision of the two persons by 
assigning weights. By a weighted combination estimate the 
length of half the line from all 60 bisections, and assign a 
standard error. (The length of the line was actually 16 cm.) 

60. Indirect Determinations from Linear Equa¬ 
tions. In this case we have measurements of n linear 
functions of m unknowns, where n exceeds m. Because 
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of observational error the equations are inconsistent. For 
example, we might have 


Observations. Weights. 

X = 1-76 2 

a?+2/ = 3*10 1 

»+2/+2 = 3*85 4 

y+z-fw = 4*30 2 

z-\~u = 3*06 3 

u = 2-10 1 


. ( 1 ) 


In such a case the method of Least Squares consists 
again in taking as optimal values those for which the 
sum of weighted squares of residuals is a minimum, so that 
for example to solve the equations (1) we would minimize 
= 2(a?~l*75)H(iC+y-3*10)2-f4(a;+t/+2-3*85)a (2) 
+2(t/+z+ii~4*30)* + 3(z4-u-3*05)*+(w-2*10)2 


with respect to Xy y, z, u. More generally, if the equations 
are (to take the case of 4 unknowns) 

ayX + b^y + CjZ + d^u = weight 
~ ^2> • • * ^2> 

a».a:+6mJ/+Cm2+d„M = h„, ... 
we minimize 

tn 

= ZwjiafX+b^y+CjZ+djU-’hf)^, . 
i=i 

and similarly for any number of unknowns. 

The partial derivatives dS^jdXy dS^ldyy dS^ldz, dS^jdu 
must be zero ; and so we derive the equations 

{Ewfi^^x -\-{Ewfl>jbj)y-\-{Ewfbfif)z-\-{Ew^^dj)u=Ewfljhjy 
(Ewfljbi)x-{-[Ewjbi'^)y ■\-(Ewjbfi)z--\-(Ewf)^j)u—Ewjbjhiy 
{Ewflfii)x-^(Ewf)fij)y-\-(Ewfi^)z -\-(Ewf^dj)u^Ewfjhjy * * 
(Ewfl^i)x~\r{Ewibfij)y-{-{Ewf^^)z-\-{Ew^^)u ^Ew^d^hj, 

for Xy y, Zy u. These are called the normal equations, and 
their general form is similar to the above. Inspection will 


. (3) 

* ( 4 ) 
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show that the coefficients in the normal equations are 
symmetrical, in the respect that the coefficient of the 
unknown in the equation is identical with that of the 

unknown in the equation. The scheme of coefficients 
ip in fact symmetrical about its northwest-to-southeast 
diagonal. This symmetry is of great service in shortening 
the solution of the equations. 

Thus in our numerical example the normal equations will 
be found to be 

Tx-f 5 i/+4z = 22*000, 

6a; + 72/ + 62+2w = 27*100, . . (6) 

4a; + 67/ + d2 4-5w = 33*150, 

22/+5z-f6t^ = 19*850, 

which can now be solved by methods of practical algebra. 
The solutions are x — 1*750, y = 1*274, z = 0*846, u = 2*178. 

Various schemes of systematic solution of normal 
equations have been devised, and for these the reader 
must be referred to more comprehensive treatises and 
original memoirs dealing with Least Squares or with the 
numerical solution of algebraic equations. 

Preparation of Normal Equations. It is evident 
from the construction of the sum o^ weighted and 
squared residuals that exactly the same sum would arise 
if we multiplied each observation throughout by the 
square root of its weight, and then regarded the 

observational equations as of equal unit weight. (Let the 
reader verify this from the example.) Such a reduction 
of a set of equations with unequal weights to a set with 
equal weights is called preparing the equations. 

61. Application of Least Squares to Trivariate 
Correlation. Suppose that we have n trivariate obser¬ 
vations (Xj, y^, Zj), as for example the height, weight and 
chest measurement of each of 1000 soldiers, and that we 
wish to express each variate as the best possible linear 
estimate of the other two. We may suppose the variates 
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measured as deviations from their respective means, and 
standardized. Thus for x we have n equations 

== ^i 22 /i+ 6 i 3 Z^, (j = 1, 2, 3, n). . (1) 

These may be regarded as n observational equations in 
the two unknowns solve them by least 

squares we shall have the desired optimal relation 

^ • • • ( 2 ) 

which may be regarded geometrically as the regression 
plane o^ x on y and z. The coefficients ^13 ^.re 

called regression coefficients ; they are the sample estimates 
of ideal regression coefficients j 8 i 3 in the underlying 
population. The normal equations for and 6^3 are 
obtained by minimizing the sum of squared residuals 

, . (3) 

i 

The minimum conditions dS^jdb^^ =0, dS^jdb^^ =0 
give, on division by n, 

^ 12 +^ 23^13 = ^ 12 > ^ 

^23^12+^13 = ^13> 

where r^^^^ Zx^y^jn, r^^^Exf^jn, r^^^Ey^z^jn, 

Solving, we find the desired regression coefficients as 

^12 = (^12~^13^23)/(1~^’23)» ff'K 

^13= (^13““^12^23)/(1“^2^3)> 

and similar results hold for the regression of y on a; and z, 
and of z on a; and y. 

The standardized mean-product-deviations r^^ and 
fgg are usually called total correlation coefficients of x and 
y, x and z and y and z respectively. They are really 
estimates from sample of the corresponding mean-product- 
deviations, or product-moments Pi 3 and p^^ in the 
trivariate population or probability function. 
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It may be proved that the trivariate normal m.g.f., 
in standard scale and with means as origin, is 

expl{a^+2 +2 pi^ay+2 p^aPy) . (6) 

and by reversion that the corresponding trivariatc normal 
function is 

y, 2 ) 

= (2^)-#A-iexp [-iA’Hil -p|3)a;M-(l -PiVy*+(l-/>i|)2* 
^(Pli ftsP23)*y ^iPia Pl2p2a)^^~ 2{p23 ft2ft3)y2}]> (7) 

where A is the determinant 

1 Pl2 Pl3 
A = pi 2 1 P 23 

Pl3 P23 ^ 

of total correlations. 

The equations d<f)ldx = 0, d(l>ldy = 0, d<f>ldz = 0 give 
the loci of maximum probability of x for fixed y and z, 
of y for fixed x and z, and of z for fixed x and y. By 
actual differentiation we find these loci to be 

* “ • • (^) 

and two others, where 

Pl2 — ipl2~Pl3P23)l(^~~Piz)> /Q\ 

Pl3 ~ (Pl3~Pl2P23)/(^—pi})' 

Thus we see that the estimates of regression by Least 
Squares are in agreement with those based on normal 
trivariate correlation. A corresponding result is true for 
linear regression in any number of variates. 

62. Partial Correlation. The unstandardized equa¬ 
tions, with means as origin, of the regression lines in 
bivariate regression (49) are 

X = where = p^i/crzy 

y = where ^oi = po^ja^. . . (1) 

The correlation coefficient p appears here as the 
geometric mean On the analogy of this, partial 

H 
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correlation coefficients in multivariate problems have been 
defined as the geometric means of the corresponding 
regression coefficients. For example, the partial coefficient 
of two variables and would be defined by 

Notation. It is customary to denote, for example in 
a four-variate problem, the partial correlation coefficient 
of X and y by pi 2 » 84* ^ distinguish it from the total 
correlation coefficient pi 2 > The sample estimate would be 
written 84 * 

Example. Given the following estimates of variances and 
total correlations of three variables x, y, z, find the three 
regression equations and the three estimates of partial 
correlation coefficients; 

cl = 6-0, = 7-0, (T® = 3-0, = 0-80, = 0-40, 

= 0 . 60 . 

63. Non-Linear Regression : Polynomial Regres¬ 
sion. From the nature of a set of observations of a 
variate y dependent on x it may be apparent that the 
regression cannot be linear. Common types of non-linear 
regression are those in which the underlying functional 
relation of y and x is of polynomial, or of harmonic type. 

The polynomial regression 

y = • • (1) 

will be considered first in its simplest case, the fitting 
of the polynomial by I^east Squares to n independent 
observations of equal weight, corresponding to n 
equispaced values of x, namely a; = 0, 1, ...,w—1. 

The polynomial of best fit is given by the minimum 
of the sum of squared residuals 

Cq—...—. (2) 

X 

that is, by the conditions dS^JdCf = 0. These give ifc+1 
normal equations for the c^, easily seen to be expressible as 

Zx^(Ux—yx) = 0, (j = 0, 1, 2, . (3) 
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displaying the fact that the fitting of a polynomial of degree 
k by Least Squares is equivalent to equating the moments 
of orders 0, 1, 2, & of the polynomial and the data. 

The values of the coefficients can be found by solving 
the normal equations, but since sums of powers of natural 
numbers up to the are required, the method becomes 
laborious if n is large and if the polynomial y is of the 
3rd or higher degree. For this reason it is better to 
express y not in powers of x, but in polynomials 
1, t^{x)y < 2 (^)> •••» having the property of being 

uncorrelated, being in fact such that the product sum 

Eti{x)ti(x) = 0 ifi . . . (4) 

X 

These polynomials tj{x) are familiar in mathematics as 
the orthogonal polynomials of Tchebychef, and their 
properties are known. For example, it is known that 

= (2j)(«a;,„-(2j-l),,,(w - 

4-(2j—2)(y)(n—y(n—l)(^), . (5) 

SO that (Appendix, 1) the difference 
/\%{x) = (2j)„-,x„_„-(2j-l)(,-,(w-j).r„_,_i, 

It is also known that 

SUx))^ = n(n*-l)(n®-4)...(«*-iM(2i4-l)(j!)*]. (7) 

X 

If, therefore, we express y in the form 

2/ = ao +<hk{^) +«2^2(3?)+• • • » • • (8) 

the sum of squared residuals, because of the vanishing 
of the product terms, takes the form 

X 

= ^ [K)^-2w*(ao +ai<i(a;) +... +«*<*(*)) 

+Oo+• • • + • (9) 
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and the normal equations dS^jda^ = 0 therefore take the 
form 

a,^Su^t,(x)lSUx))K (j = 0, 1, A;). . (10) 

Thus each coefficient in the regression is found independ¬ 
ently of the others, without the labour of solving 
simultaneous equations. (The choice of uncorrelated or 
orthogonal functions for the representation of Ux always 
confers this very great advantage.) Since the polynomials 
ti{x) are expressed in factorials x^^), the numerator of the 
expression for can easily be found in terms of the 
factorial momenta of the data Ux, these moments being 
obtained as usual (Appendix, 2) by summation. 

The minim um sum of squared residuals can itself be 
evaluated beforehand, for by (9) and (10) it takes the form 

= ( 11 ) 

mvolving the sum of the squares of the Ux, diminished 
by the product of each successive by the numerator 
in (10). It is known that the variance of a single residual 
is best estimated by dividing the sum of the n squared 
residuals by the degrees of freedom, n—k—l ; hence we 
can judge beforehand, if we know the precision of the data, 
what value of k gives the best polynomial y. It is of 
course possible, by taking too many terms in the polynomial 
y, to fit the data too well, in the sense that the sum of 
squared residuals is much smaller than that warranted by 
the precision of the data. 

64. Practical Routine of Fitting a Polynomial. 

All of the above points, which can be treated only briefly 
here, have been discussed at length in special memoirs. 
We shall merely illustrate a method depending on the 
theory of 63 and making use of a table containing the 
terminal values and differences 1^(0), Atj{0)y A%(0), ..., 
for J= 0, 1, 2, 3, ..., k and the particular value of n. 
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The rule for constructing such a table follows from (5) 
and (6) and is simple. We shall illustrate it for w = 6. 
We write down the fixed table of binomial coefficients, 
table (i) below, to columns ; in the illustration, k ~ 3. 
Beside table (i) we place table (ii), consisting of binomial 
coefficients of w—1, n—2, ... written below each other as 
shown, also to k-\-l columns. The products of corre¬ 
sponding entries in the two tables now give us the desired 
table (iii) of terminal values and differences of ^polynomials, 
and at the feet of the respective columns we enter the 
values of as computed from the formula 63 (7). 


1-1 1-1 
2-3 4 

6 -10 
20 

( ii ) 

1 6 10 10 

1 4 6 

1 3 

1 

1 -6 10 -10 i 

( iv ) 

1-5 6-5 

2 -12 24 


2 -6 12 

6 -30 


3 -15 

20 


10 


6 70 336 720 6 70 84 180 


A possibility making table (iii) still simpler for practical 
use is that when a common integer factor is observed in 
any column, we may cancel through by that factor, 
provided that the square of that factor is cancelled through 
from Thus the cancelling of factors 2, 2 from columns 
3, 4 in table (iii) above gives table (iv). Such tables, 
extended to six or seven columns, are easily constructed 
for a proposed value of n. 

The use of the table in finding the regression coefficients 
and the fitted values is best illustrated by an actual 
worked example. The process is no more difficult for a 
long series of data than for a short, but to economize in 
space we shall illustrate it by fitting a cubic polynomial to 
six values 
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Example. 

a? 0 1 2 3 4 6 

w 5 13 25 60 105 200 

By summation the reduced factorial moments of u are 
found to bo 408, 1663, 2835 and 2480, while ~ 55444. 

Using four columns only of the table of polynomials (since 
we are fitting a cubic) we set out the rest of the work in 
compact shape thus ; 

408 1286 567 191 

68 18-371 6-75 1-0611 A^o AVo 


4-590 

8-975 

4-334 

10-611 

6 70 84 180 Check = 198-91. 

Explanation^ 

ao = (408xl)/6 = 68. 

Oi = (1663 X2-408 x5)/70 = 1286/70 = 18-371. 
da = (2835 X 3-1663 X 6+408 X 5)/84 = 567/84 = 6-75, 

and so on ; the elements in columns of the table are used as 
multipliers of the factorial moments, the entries at the feet of 
the columns as divisors. Then 

2/0 = 68x1-18-371x5+6-75x5-1-0611x5 = 4-590. 
At/o = 18-371x2-6-75x6 + 1-0611x12 = 8-975. 

A" 2 /o =» 6-75x3-1-0611x15 = 4-334, 

and so on ; the elements in rows are now used as multipliers 
of the tty, and give the terminal value and its differences. 
There is also a good check on the other terminal value, 

2 /a =: 68x1 + 18-371x5+6-75x5 + 1-0611x5 = 198-91, 

the same terms as gave 2/o» but with positive multipliers. 

Building up a difference table of the from the constant 
3rd differences in the way familiar in interpolation, we have— 


Sums 

1-6 6 -5 

408 

2 -6 12 

1663 

3 -15 

2835 

10 

2480 
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X 

y 

Ay 

A^y 

A^y 

u 

u—y 

0 

4*590 

8*975 



5 

0*410 

1 

13*565 

13*309 

4*334 

10*611 

13 

-0*565 

2 

26*874 

28*254 

14*945 

10*611 

25 

-1*874 

3 

55*128 

53*810 

25*556 

10*611 

60 

4*872 

4 

108*938 

89*977 

36*167 


105 

-3*938 

5 

198*915 




200 

1*085 


The comparison of the fitted values with the data can be 
seen in the columns headed y and u. The sum of squared 
residuals (u—yY will be found to be 44*4. 

But we can also set out a table thus, estimating by 63 (11) 
the variance of a residual after a constant, a straight line, 
a parabola and our cubic are fitted in succession : 


k 

n—A; —1 


num. of a* 

prod. 

55444 

1 

1 

0 

5 

68 

408 

27744 

27700 

5540 

1 

4 

18*371 

1286 

23626 

4074 

1019 

2 

3 

6*75 

567 

3827 

247 

82 

3 

2 

1*0611 

191 

203 

44 

22 


The column headed shows the sum of squared residuals, 
obtained in accordance witli 63 (11) by subtracting the 
entries in the previous column in turn from = 55444. 
The last column gives estimates of the variance of a single 
residual at the different stages. To test which polynomial 
best represents the data, we must have a preliminary 
knowledge or estimate of the variance of the observations. 
This variance is compared with the residual variance in 
the light of the sampling distributions of 71 and 74. 

The alternative computation of the sum of squared residuals 
as 44 checks the work, for the same sum was given by the 
fitted values as 44*4. 

For a given value of n the same table of terminal values 
and differences of ^-polynomials serves for fitting a poly- 
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nomial of any degree. Thus, using only the first three 
colunms of the table in the above worked example, we 
may fit a parabola instead of a cubic. It will be found 
instructive to do this, following the details of the worked 
example. Notice that the coefficients 0 ^, ag are the 
same as before. 


Example. Fit a cubic polynomial to the seven equidistant 
and equally weighted data 

X 0 1 2 3 4 5 6 

u -11 6 13 25 60 105 200 

65. Periodic Regressions : Observations of Equal 
Weight. Observations which exhibit periodicity more or 
less masked by accidental error are of common occurrence. 
The height of tide-water at a seaport, measured at equal 
intervals of time, shows such a periodicity ; monthly 
averages of temperature show a seasonal periodicity; 
telephone calls on an Exchange show a weekly periodicity. 

The procedure for analysing periodicity is to assume a 
periodic function 

_ -\-(iiCOs0 -fagCOBSO +.,. cos kO . 

^ + 6 isin 0 +62 sin 20 - 1 - 6 ;^ sin kO 

and to find the coefficients and of the constituent 
periodic terms by the method of Least Squares. 

We consider therefore n equally spaced observations 
U 0 of equal weight, where 0 = 0 , 27r/w, 47 r/n, ..., 2 (n—l) 7 r/n, 
the observations thus corresponding to the n phase-angles 
of one complete oscillation of a periodic phenomenon. 
The initial observation of a second oscillation is not 
included. In view of the trigonometrical relations 


„ , ( 2 ) 

n n = in, if h = 


and the similar ones with one or both cosines replaced by 
sines (these are really orthogonal relations exactly 
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resembling those of the Tchebychef polynomials in 63 (4) 
and (7)), the sum of squared residuals is (cf, 63 (9)) 

== Z[u^Q— 2 uQ{aQ+a^ cos 6 +...+bj^ sin kO)] 

0 6 

+|n(2a2+c^+...+fe|). . (3) 

Differentiating with respect to the and bf and 
equating to zero, we have the normal equations for the 
regression coefficients. Each is given independently of 
the others. 

12 2 
Uo = —Suay a* = • (^) 

n a ^ 71 a ^ fla ^ 


If n is even, 

o,,, = cos h\B, 

* n 0 ^ 


6j„ ^ 0, 


. (5) 


and cos is +1 and —1 alternately as 0 takes its n 
values. 

The theoretical solution is thus immediate. Simplicity 
of practical application will depend on the value of n, and 
the consequent values of cos hd and sin hd. 


66. Practical Solution of the Normal Equations. 

The process of numerical solution becomes specially simple 
when 0, that is, 27r/7i, is such that cos hO and sin 7i0 are 
easy to handle. This occurs when n = 4, 6 , 8 , 12 or 24, 
the last two cases being specially important, as corre¬ 
sponding to the hourly or two-hourly subdivision of the 
day ; and special routines for these values of n have been 
devised. 

The procedure depends on the fact that in the four 
quadrants, from 0 = 0 to 0 = 27t, cos 0 and sin 0 take the 
same absolute values four times, though with differing 
alternations of sign. To take the case n = 12 for illus¬ 
tration, the data U 0 (and there will be no misunderstanding 
if these are written meanwhile as Vr^y ..., can be 
assembled in tetrads, for example before 
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being multiplied by the suitable values of cos^ and 
sin hJ9, where hO can always be taken as coterminous with 
some angle in the first quadrant. 

We shall indicate how this is done by an actual example. 

Example. To fit terms as far as a 40 os 4^, 64 sin 4^ to the 
12 data (Whittaker and Robinson, Calculus of Observations^ 
p. 272 ): 

Wq Ui Wj W3 W4 W5 Wfl W7 Wg ^9 *^10 **11 

2*71 3*04 2-13 1*27 0-79 0*60 0*37 0-54 0-19 - 0-36 - 0*44 0-77 


First write the data in a scheme U of columns, down, up, 
down, up, with blanks as indicated by the dots, as follows : 



~271 

37 

. 




+ 

4- 

4- 

u = 

304 

50 

54 

77 

M = 

+ 

— 

— 

4- 

213 

79 

19 

-44 

4- 

— 

+ 

— 


127 

• 

-35 

• _ 

» 

■f 

4- 

— 



Next, add along the rows of the scheme U, after giving 
sign to the columns of U in four different ways, according 
to the rows in the sign-scheme M, We thus obtain four 
separate sets of totals, and these are combined with cosines 
and sines of 0®, 30®, 60®, 90® in four separate schemes, as 
below. (We have included the coefficients necessary for 
computing O 5 , 65 and O 4 as well.) 




U 2 

04 



«i 

Os 

Os 

308 

0*5 

1 

1 

0*5 

234 

1 

1 

1 

485 

0*5 

0*5 -0*5 

-0*5 

277 

0*866 

0 

-0*866 

267 

0*5 - 

-0*5 -0*5 

0*5 

71 

0*5 

-1 

0*5 

92 

0*5 - 

-1 

1 

-0*5 

162 

0 

0 

0 

6 

576 325 

24 

-1 


509*4 

163 

26*6 

0*960 0*542 

0*040 

-0*002 


"0-849 

0*272 

0*044 


h 


b. 



bi 



234 

0 


0 


308 

0 

0 

0 

231 

0*866 


0*866 


223 

0*5 

1 

0*5 

197 

0*866 

— 

0*866 


317 

0*866 

0 

-0*866 

92 

0 


0 


162 

1 

-1 

1 

6 

370*6 


29*4 


6 


548 

61 

-0*6 


0*618 


0*049 



0*913 

0*102 

-0*001 
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Explanation, 

Oi =» (2-34Xl-h2-77x0-866-f 0-71 x0*64'1-62x0)/6 

= 0*849, etc. 

Hence, as far as terms in cos 40, sin 40, the regression is 
4-0*849 cos 04-0*642 cos 204-0*272 cos 30 
= 0*960 +0*040 cos 40 

+0*913 sin 0+0*618 sin 20+0*102 sin 30 

+0*049 sin 40, 

and the regressions to fewer or more terms involve the 
same coefficients a^, 6^ as are given by the above scheme 
of solution. 

The sum of squared residuals may also be calculated 
beforehand from the regression coefficients in a scheme set 
out as follows ; 


na^ and 


k 

n-2Jfc-l 

Jn(a|+62) 

aS* 

24*983 

+(n—2A; —1) 

0 

11 

11*059 

13*924 

1*266 

1 

9 

9*326 

4*598 

0*611 

2 

7 

4*054 

0*544 

0*078 

3 

6 

0*606 

0*038 

0*008 

4 

3 

0*024 

0*014 

0*005 


Just as in polynomial regression, the contributions to the 
sum of squared residuals produced by successive terms are 
subtracted in turn from which here is 24*983. The estimate 
of variance of a single residual is then made by dividing the 
residual sum of squares by n—2k—l, the number of degrees 
of freedom. The results are shown in the last column. 

67. General Regressions. After what has preceded, 
the routine to be adopted in other regressions, such as 

y = a^4-ai tanfl+agtan 20+...+a;fctan . (1) 

will be readily understood. Such regressions are not 
common in statistical work, but they are not outside the 
bounds of possibility. The desirable thing in any problem 
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of regression will be to express y, if possible, in terms of 
functions which, like the Tchebychef polynomials or the 
sine and cosine of multiples of 27r/7i, have the orthogonal 
property that the product-sums of different functions of 
the set over the range vanish. The effective meaning of 
this is that the contributions of the successive terms to 
the regression are uncorrelated with each other. 

Harmonic Analysis. For fuller details concerning the 
practical routine of estimating periodic regressions, the 
reader may consult the chapters on harmonic analysis in 
Whittaker and Robinson’s Calculus of Observations, or 
Brunt’s Combination of Observations^ 2nd edition, 1931. 



OHAPTBB VII 


PROBABILITY DISTRIBUTIONS OF STATISTICAL 
COEFFICIENTS 

68. Sampling Distributions. A statistical coefficient 
computed from a sample of n values, univariate or multi¬ 
variate, is only an estimate of the corresponding parameter 
in the population or underlying probability function. It 
is therefore to be presumed erroneous, though the degree 
of error cannot be affirmed exactly, since the true value of 
the parameter is not known. The degree of error can be 
stated only in terms of probability; and the probability 
distributions involved are (i) the hypothetical population, 
or distribution of the variate or variates, (ii) the derived 
distribution of the coefficient of estimate from sample. 
The second of these is called the sampling distribution of 
the coefficient. 

Let us consider a case in which the first of these 
distributions, the probability distribution of the variate, 
is not hypothetical but given. In Charlier’s experiment 
(22) of Rawing 10 cards from a pack, with replacement 
of each card, and continuing this until a sample of 1000 
sets of 10 cards had been collected, the variate was the 
number x of black cards in a set of 10, and its probability 
distribution was the binomial distribution, with mean 5 
and variance 2*6 ; the corresponding values of mean and 
mean square deviation in Charlier’s sample were 4*933 
and 2*416. Are the respective deviations 4*933—5, or 
—0*067, and 2*415—2*5, or —0*085, reasonable or 
abnormal ? Such questions can be answered only when 
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the sampling distributions of the estimates of mean and 
variance are known. 

The nature and genesis of these sampling distributions 
can be illustrated from this same example. The sample 
group of 1000 sets of 10 card drawings was merely one 
out of an enormous number of equally possible groups. 
From the pack of 52 cards the 10 cards, drawn one at 
a time with replacement, could eventuate, if order of 
drawing were taken into account, in 52^® ways. This is 
an unimaginably large number, but the number of groups 
of 1000 sets which may be chosen from these 52^® sets 
is incomparably greater still. Each group may be 
supposed to have its mean m and mean square deviation 

computable in the usual way. The aggregates of these 
values of m and 8^ constitute probability distributions, 
and these are the sampling distributions of m and s^ for 
the kind of sample in question. 

Example. If the parent population is normal and the 
number in sample is n, the sampling variances of the 
estimates of the moments /xj, /14 are respectively 

2 a^/n, Qa^jn, 96(r*/n. For ^ 5 , /x*. ... they increase rapidly. 

The functional form of a sampling distribution depends 
(i) on the population (probability function of the variate 
or variates sampled), (ii) on the function used for estimating 
the parameter, and (iii) on n, the number of observations 
in the sample. Since 1900, and especially since 1915, 
much research has been expended on the problem of 
deriving the probability distributions of the commoner 
coefficients. Most of this research has been devoted to 
samples of a normally distributed variate or variates, and 
the sampling distributions are now well known and already 
classic. It appears that as the number n in sample 
increases the sampling distributions of many coefficients, 
though by no means of all, tend themselves towards the 
normal type. In such cases it is customary to supply an 
estimate of the precision of a coefficient by appending to 
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its computed value its standard deviation of sampling, or 
standard error ; and this is sometimes said to imply a 
probability of 19/20 that the true value lies in the range 
delimited by twice (more precisely, 1*96 times) the standard 
error on either side of the computed value. 

The form of statement is not strictly accurate ; if, for 
example, a computed mean is m, and the central 95 per cent, 
range of sampling probability area is the criterion of what is 
acceptable, then m may be anywhere from the extreme left 
of the 95 per cent, range of a sampling distribution centred 
on a hypothetical mean /x' to the extreme right of the 95 per 
cent, range of a second sampling distribution centred on a 
hypothetical mean fi" ; but these are different distributions, 
and it does not follow either that the left half-range /x'—m 
of the first is equal to the right half-range m — of the 
second, or that we can add the probabilities, under the different 
hypotheses, that the true fi lies in these respective half-ranges. 
An illuminating discussion of the problem is given in a paper 
by Clopper and E. S. Pearson, Biometrika 26 (1934), p. 404. 

When the number in sample n is small the sampling 
distribution of the coefficient is often of non-normal, skew 
or platykurtic type, and the standard error is an insufficient 
indication of the interval within which the true value of 
the parameter may lie. It is necessary in such a case to 
know the sampling distribution and probability integral 
of the special coefficient. 


69. The Sampling Distribution of Means. In a 

few cases the sampling probability function of the mean 
of n observations is of the same type as the probability 
function of the population. For example, the normal 
probability function with mean and variance o-^. 


^(x) = 


1 

‘^V'2w 




• ( 1 ) 


has m.g.f. exp (/xa+ior^a^). Hence the m.g.f. of the sum 
of n sample values is exp (nfia+ina^a^). To change 
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from sum to mean is to write xjn for x, or afn for a. Hence 
the m.g.f. of the mean of sample is exp (jita+la^a^/n). 
The mean of sample is thus distributed normally about 
the same mean as before, but with variance o-^/w, or 
standard error or/Vw. 

Ex. 1. Prove that if ... are the cumulants 

of any population, the cumulants of the mean of a sample 
of n are kx, .... 

Ex. 2. The number x of black cards in a set of 10 in 
Charlier’s experiment is binomially distributed with mean 5 
and variance 2*6. The mean of a? in 1000 sets is distributed 
with approximate normality, about mean 5, and with variance 
2*5/1000, or 0*0026. The standard error is thus 0*06. The 
deviation of the mean 4*933 of Charlier’s sample from 6 is 
—0*067, about 4/3 of the standard error. 

The deviation is not excessive. From the table of the 
normal probability integral on p. 144 it is seen that the 
probability of a deviation exceeding l*34or is about 0*18. 

Again, if the probability function of x is of Gamma or 
so-called namely 

^{x) = . . ( 2 ) 

the m.g.f. is 

/•ao 

(r(ifc))-H (l-a)-*. . (3) 

The m.g.f. of the sum of n sample values Xf is (1—a)""*, 
and so the m.g.f. of m, the sample mean, is (1—a/n)""*. 
Reverting to the probability function, which by a theorem 
of Lerch is unique, we obtain the probability function 
ofm as 

. (4) 

This is again of Pearson’s Type III. 

Ex. 3. Provo that the distribution of the sum (not the 
mean) of n values each obeying the Poissonian law ip(x) of 
33 is Poissonian. (Use the f.m.g.f. of x.) 
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70. Distribution of Mean Square in Normal 
Sample. If the probability diflFerential of x is 


dp = 


1 




that of 2 = ^x^y as in 37 (2), is 


. ( 1 ) 


dp= -4= z~^e~^dzy . , . (2) 

V TT 

and so the m.g.f. of z is (1—a)“^, by 69 (3). Hence the 
m.g.f. of half the sum of the squares of n sample values x^ 
is and so if is the mean of the squares the 

m.g.f. of ^5 2 is (1 —a/n) ~ It follows that the probability 

diflFerential of Uy where u — is 

dp = . . (3) 

again of type. By changing from to we have 
the probability function of s^y namely 

In unstandardized units we must write s^/a^ for 5^ on 
the right of (4), and insert the factor l/cr^. 

The seminvariant g.f. of 5^ is 

—\n log (1—2a/n) — \n{2ajn-\-4<x^j2n^ + ,..) 

= a+2a 2/2171+.(5) 

Thus the mean of is 1 and the variance is 2/n ; in 
unstandardized units these are 0-2 and 2o*‘jny where a 2 
is the variance of x. The s.g.f. also shows that as n 
increases the m.g.f. of s* tends to asymptotic equivalence 
with exp (a+a^/n) ; and so the distribution of 52 tends 
to normality. 


Example. The distribution of 5 ® in Charlier’s 1000 sets is 
almost normal; a* = 2*5, and computed from the sample 
(using deviations not from Charlier’s mean m — 4*933, but 
from /i = 6) is 2*419. The standard error of is a®\/(2/7i) 
= 2*6/VSOO = 0*112. The actual deviation, —0*081, is 
nmnerically about three quarters of this. 


I 
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71. Distribution of Estimate of Variance. The 
variance or second moment of the population is commonly 
estimated from the sample by taking the part of the 
sum of squared deviations of the sample values from 
the sample mean m. 

Of the n deviations from m only n—l are independent, 
and this estimate of which we shall call though 
pointing out that it is not the same 8^ as in 70, can be 
expressed as a quadratic expression in w—1 independent 
values. Thus we have, by 14 (5), 

= (*f+a|+...+a:2)/n-(a;j+a:2-f-...+a;j2/»* 

= (n—l)(zl+zl+...+zl_j)ln^ 

-2(ZiZ 2+V3+—+2n-2*n-l)/»»^ • . . (1) 

where % = Xi-x„, z^ = x^-x^, z„_i = ; and 

this is but one of many ways in which may bo expressed 
in terms of only n—1 variables. The here are linearly 
independent, though correlated by possessing the term 
in common. (See Appendix, 5.) 

This loss of a degree of freedom, for that is what it is, 
complicates the problem of finding the distribution of 5 2 , 
but its m.g.f. can be evaluated as a multiple integral over 
the n sample values, and proves to be (1— 
which differs from that of the in 70 only in the exponent, 

n—\ replacing n. It follows that the distribution of 5 * is 
again of type, its probability function being in fact 

(2) 

which should be compared with that of 70 (4). 

This distribution is called Helmert’s distribution, after 
the German astronomer and geodetist F. R. Helmert, who 
published it in 1876. 

By expanding the m.g.f. and noting the coefficient of 
a we find that the mean value of over all samples of 
n is {n—l)(]i^lnf where a* is the variance of x. This 
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is really the theorem of mean square residual of 69 (5), 
and it is true not merely for normal but for general popula¬ 
tions. Because of the factor (n—l)ln the precept is often 
given to estimate variance by dividing the sum of squared 
deviations from m not by n but by w—1. On the other 
hand, the discrepancy in caused by not doing this is 
of order l/n, whereas the standard error of sampling of 5 ^ 
is of order \/{2ln). Thus the error of method is to the 
error of sampling in approximate ratio 1 : which 

even for n as small as 25 is less than 1/7. To insist on 
the divisor n—l rather than n in large samples may 
therefore seem a little pedantic; but in small samples 
an appreciable difference is made. One advantage of the 
division by n—l is this, that with the modified the 
probability function (2) assumes the form 

which is now of exactly the same form as in 70 (4), with 
n—l for n throughout. Thus the loss of a degree of 
freedom is made apparent. In unstandardized units we 
must write for a^ and insert on the right of (3) the 
factor l/or*. 

The m.g.f. of in (3) is [1—2a/(n—from 
which it follows, as in 70 (5), that the sampling variance 
of this modified is 2(T^/(n—1), the standard error thus 
being a^^/2|^/{n—l), 

Example. By considering the coefficients of a®/3! and 
a^/4! in the s.g.f. investigate the skewness and excess of the 
distribution of 5*. 

72. '' Student’s Ratio ” t and its Distribution. We 
have seen in 69 that the mean m of a sample of n values 

drawn from a normal population of mean /x and variance 
or^ is distributed normally with mean ji and variance a^jn. 
It follows that the standardized deviation (m—/x)\/n/flr is 
distributed normally with mean zero and variance 1. 
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Now in practice we do not know a® and so we cannot 
standardize the scale. All that we know is the estimate 
(taking n—1 for divisor) = Z{Xf—in)^l{n—l), The 
deviation of the mean of sample from true mean, standard¬ 
ized by this estimate is thus = t. This 

is “ Student’s Ratio,” and it is not normally distributed. 

“ Student ** was the pen-name under which W. S. Gosset 
(1876-1937) wrote his statistical papers. He discovered the 
distribution in 1908. 

To simplify the distribution we may place the origin 
of X a,t X = jjL, thus putting fz = 0. Then mVnja == t. 
Since mV nja = (mVnlG)l(ala), and since the distributions 
of mVn/a and a^/o^ are independent of or, we may use 
standard scale with o = 1. 

For conatant a^ we have dm = adt/V'^ *» 
probability of obtaining the value t is the probability that 
m takes the value atjV'^^f and the probability differential 
for this is 

ce’“^^^'^dm = . . ( 1 ) 

This is for constant a^ ; and so the probability differential 
of t is the integral of (1) over all values of a^. Hence, 
multiplying (1) by the probability differential of which 
we already know from 71 (3), and integrating from 0 to oo, 
we have 

/•oo 

dp{t) = Cidti 

= . . . ( 2 ) 

where c, = ^0) | . (3) 

the constant Cg being fixed, as always, by the condition 
that the total probability is 1. 

Note. The above derivation is the one usually given, 
but an important remark must be made. The essential step, 
the compounding of the probability differentials of m and a^, 
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pres'umes the statistical independence of m and This 
independence (Appendix, 5) is not evident, nor is it 
capable of quite elementary proof. The reader may assume, 
however, both here and in the case of the difference of means 
of two samples, that the numerator and denominator of t 
are independent. 

The remarkable and important fact about the 
^-distribution is that it does not involve the unknown 
a partial reason being that < is a ratio, of zero dimension 
in (7 2. The discovery of the distribution in 1908 had a 
profound influence on “ small sample theory; for 
whereas it had long been conventional to take s as the 
presumptive a and to estimate the probable region of the 
unknown ji by regarding {m—fjL)Vnls as a standardized 
normal variate, this was now seen to be an inexact 
procedure, and the ^-distribution was used instead. 

Since tends with increasing n to 

exp it is apparent that for large samples the 

^distribution tends to the standard normal one; but the 
tendency is not rapid, and for small values of n, as one 
might suspect from noting that n — 2 gives the Cauchy 
distribution, the departure from normality is marked, the 
curves being platykurtic. For example, whereas in the 
normal curve 0*95 of the area is contained in the range 
X = —1*96 to x = 1-96, in the ^-curve for n = 10 the same 
area lies between t = — 2*26 and i = 2*26 ; and for area 
0*99 the ranges are given by x — :^2'5S and t = ^3-25. 

A table of the probability integral of the ^-distribution, 
in a form useful for practical application, is given in R. A. 
Fisher’s Statistical Methods far liesearch Workers^ 8th edition, 
p. 167. His n is our 7^—1, the number of degrees of freedom. 

Example. A coin, thrown 20 times on each of 10 occasions, 
shows 7, 9. 6, 10, 13, 6, 9, 7, 10, 7 heads respectively. 
Assuming the binomial distribution of 20 throws to be 
approximately normal, consider v hether the coin is biassed. 

The mean of the heads thrown is ni — 8*4 and = 4* 93, 
fi = 2* 22. Thus, presuming an unbiassed /n = 10, we have 
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t = (10--8‘4)^10/2*22 = 2*28. From Fisher’s tables, in the 
row n — 9 (our n—l) we find t = 2*262 at P = 0*06. (P is 
the probability of a ^ numerically greater than 2*262.) Thus 
the coin-throws leave it rather doubtful whether the coin is 
biassed or not. 

A reading of tables of the normal probability integral for 
a? = 2*28 would have given P = 0*023, with an unjustifiably 
stronger suggestion of bias in the coin. 

73. Difference of Means of Two Normal Samples. 

A valuable use of the ^-distribution is in testing the 
hypothesis that two samples, with different numbers n 
and N in sample, are from the same normal population, 
of mean p, = 0 and variance a*. 

Let X 2 , Xn be the first sample, X^, Xg, 
be the second, with respective means m and M, and 
estimates of variance 


The basis of the test is the difference the 

variance of which (15) is 

a^ln+Q^jN = {n+N)a^lnN, , . ( 2 ) 

The estimates and 8^ of are (71) of weights n—l 
and X—1, and so yield a combined estimate of a*, namely 


82 = [{n-l)s^+{N-l)8^]l{n+N-2) 

= [E(x^-m)^+E(Xf-M)^]l{n+N-2). . (3) 


It can be proved that m—M and s* are statistically 
independent. We therefore define, from (2) and (3), 


m—M [ nN \ 


• (4) 


and it now follows, by the argument of 72, that this t 
has the ^-distribution, but with n+X—2, the number of 
degrees of freedom used in estimating a*, in place of the 
former n—l. Thus the ^tables may be consulted for the 
probability P{t) that t numerically exceeds any assigned 
value. (The examples of p. 109 are amenable to ^tests.) 
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The important point is the way in which a® is estimated. 
One might have pooled both samples and estimated a* from 
the squared n 4*-^ deviations about the pooled mean 
(nm-\-NM)l(n-\-N)y summed and divided by n+JV —1. This 
slightly more accurate estimate of a* is, however, not 
independent of m—M. 

74. The Ratio of Two Variates of the Same 
Type. The two samples in 73 give in general different 
estimates 5^ and of the variance g^. If the question 
is whether both samples are from the same normal popula¬ 
tion, we shall wish to test this by means of 5 ^ and 
without reference to the unknown The analogy of 
t suggests the ratio u = s^jS^, Since u is unaltered when 
we write s^ja^ for 5^, and S^jo^ for we may work in 
standard scale, using the 5 ^-distribution of 71. Let us 
write V = V = Then u — vjV, or v = uF. 

By 71 the ])robability differentials of v and V are 

c^vHn-z)^-Hn-i)V(iv and . (1) 

For fixed V we have dv = Vdu ; so, integrating for all V, 
we have the probability differential of Uy 

c^C^du f 

Jo ^ 

= CMJ<'‘-%M/(iV-l+n^ . . . (2) 

where c is fixed by making the integral of u unity. 

The distribution of u is thus given by (2). It is 
interesting to verify that as N—>oo the distribution tends 
to the fyp©> while if n = 2 we have a distribution. 

The z-Distribution of Fisher. R. A. Fisher, in 
testing the difference of two estimates and uses 
not this ratio u but half its natural logarithm. If we put 

z — i logtj-w, u = e“=, da 2e“*dz, 
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the probability diflFerential (2) becomes 

dp = , (3) 

where Cg is such that the total integral of z is unity. The 
distribution thus obtained is Fisher’s z-distribution. 

Tables of P(z), the probability of a z greater than an 
assigned value, are given in Fisher’s Statistical Methods for 
Research Workers, In these tables the numerator of u is 
the greater of s^ and S^, so that z is positive ; and the 
functions tabled are the values of z for assigned n and N, 
such that P = 0*06, 0*01 and 0*001 respectively. The table 
for P = 0*001 is due to C. G. Colcord and L. S. Deming. 

75. Analysis of Variance and of Sum of Squares. 

The basic idea of the experimental designs introduced by 
R. A. Fisher, and of the accompanying technique called 
analysis of variance^ is that of dividing up a total sum of 
squared deviations of a variate from its sample mean into 
several distinct sums of squares, each corresponding to a 
source, real or suspected, of variation. These partial sums 
yield estimates of the variance from each source, and the 
z-test is applied to ascertain whether these estimates are 
compatible with each other and with the estimate of residual 
variance. If they are not so compatible, it is presumed 
that the sources have distinct effects, which are further 
analysed, for example by difference (73) of means. 

The resolution into sums of squares is founded on the 
Lemma, noted in 62 in connexion with the correlation 
ratio, that if k sets of % observations, with 

respective means and mean square deviations are 
pooled in an aggregate of n = Wi+7i2+... H-w* observations, 
with mean M and mean square deviations 8^, then 

nS^ = . . . ( 1 ) 

i 

where 

For illustration we shall consider an experiment based 
on repeated trials and designed to ascertain (i) whether h 
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varieties of a cereal are different in crop yield, (ii) whether 
k kinds of fertilizing treatment are different in their effect 
on the crop yield of the h varieties. 

Consider first the case (i), the experiment on varieties 
alone. Suppose each of them planted in k similar plots, 
assigned in random positions in a field, and subjected to 
uniform cultivation. The hk yields where i refers to 
variety, j to plot-number, may be arranged for analysis 
in a rectangular scheme of h rows and k columns, a row 
to each variety. For convenience in the algebra let us 
choose the origin of so that the sum or mean of all 
is zero. 

Now consider the sum SS over all hk deviations. 

i 3 

Let the means of rows (varieties) be yjo, y 2 o» •••> ^ao* 
Then by (1), remembering that the general mean is zero, 
we have 

= • • ( 2 ) 

i } a i 

The sums here are sums of squared residuals, and under 
the assumption that all plot-yields have zero mean and 
variance a 2 , the mean values or expectations of the terms 
give, by 59 (5), the relation 

(hk-\)a^ = h(k-\)(j^+{h-l)a^,, . (3) 

where the terms correspond to those in (2). The first 
term on the right follows from the fact that the mean 
value or expectation of sum of the k squared deviations 
for any row is (fc—1 )ct^ ; and the second term then follows 
by subtraction. 

The coefficients in (3) are really degrees of freedom ; 
and we thus distinguish hk—l degrees of freedom for all 
hk plots, of which ^—1 are for variation between means 
of rows, that is, between varieties, and 1) are for 
variation about the particular variety means that is, 
within varieties. 

If the hypothesis to be tested is that varieties are not 
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essentially different in yield, this is the same as to suppose 
that variation between varieties is subject to the same 
cause as variation within varieties, that is, to ordinary 
randomness arising from soil heterogeneity and other 
causes common to all plots. The test is therefore to com¬ 
pute an 5 2 from the sum of squares between varieties 
and an from the sum of squares within varieties, these 
being independent estimates of o-^, and to see from the 
21 -table whether they are compatible. In the calculation 
of 5 2 and the respective degrees of freedom should be 
used as divisors ; and 8^ is most easily calculated by 
means of 

h{k-l)8^ = . . . (4) 

i j i 

76. Analysis into Two Sources of Variation and 
Residual. Next, still with the same A-by-i arrangement 
(which in the random placing of plots in the field is 
called the “randomized block” arrangement), let the 
rectangle of h rows and k columns of yields be set out 
for analysis in the case when there are not only h different 
varieties, but each is subjected to k different treatments, 
so that is the yield of the variety under the 
treatment. Let the means of columns (treatments) be 
yoi> ^02* •••» VoTc' 

Consider the term in 75 (2), and imagine ^ 

ij 

all the deviations from mean of variety, ya—yi^f to be 
set out in a rectangle just as the yi^ were. Since 

Eyi^ = EEy^jjk = 0, the means of the yi—y^^ in columns 
i i j 

are merely those of the y^^ themselves, namely 

•*•9 Vok" 

Hence, by analysing this term exactly as EEyf. was, 

a 

but with respect to column means instead of row means, 
we have 

^^{yij-yiof = ^^yij-yiD-yo}f+^^yli- • (i) 

a ij j 
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Hence again, from 75 (2), 

SZy% = S2:{yij-y^-y^f+kEy%+hSylj, . (2) 
i j i j i j 

which exhibits a threefold dissection, the last two terms 
on the right corresponding to variation between varieties 
and between treatments respectively, and the first term 
to residual variation. As for degrees of freedom, by taking 
expectations as before of these sums of squared residuals, 
we have 

= (h-l)(k-l)a^+(h-l)G^+[k-l)a^, (3) 

the coefficients giving the desired divisors of corresponding 
terms on the right of (2), for estimates of variance. The 
comparison of estimates by the z-table is then available. 


77. The Latin Sq[uare. In the arrangement shown 


A 

B 

G 

D 

E 

on the left, each of A, J5, C, D, E 
appears exactly five times in rows 

E 

G 

A 

B 

D 

and columns of a square, but 

B 

D 

E 

G 

A 

no letter occurs twice in the same 

D 

E 

B 

A 

G 

row or same column. Such an 

C 

A 

D 

E 

B 

arrangement of h letters each 

square 

of order h. 

repeated h times is called a Latin 


Imagine the Latin square to be a scheme of plot-yields 
set up for analysis, the letters representing yields of 
different varieties, the rows corresponding to varied 
treatments of one kind, the columns to varied treatments 
of another kind ; for example, two kinds of fertilizer 
applied at once, at h different levels of strength in each. 
There are thus three dimensions of variation, two for 
treatments and one for variety ; and so the yields may 
be written where i refers to row, j to column, I to 
variety. Let the respective means for rows, columns and 
varieties be and yQQ^. Each suffix runs from 1 to h. 
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A first analysis as in 76 (2) gives us 

ij ij i j 

But now arrange the Viji—yioo'-yojo ^ rows, say, according 
to Z = 1, 2, ..., hy and analyse once again. Since the 
means of 2/oio zero, the means of yiji^yioo 

—yoio are simply yoo/> where Z = 1, 2, A. We therefore 
have 

= ^^yiji-ym-yojo-ym)^+^^ym+^^yljo+^^ywi ( 2 ) 

ijij i j I 

where the three last terms on the right are sums of squares 
for variation between rows, columns and varieties respec¬ 
tively, and the first term is for residual variation. By 
taking the expectations of these sums of squared residuals 
we have 

(A2_i)^ 2:^ (A_l)(^_2)cr2+(A~l)a2+(A-l)a2+(A-l)a2, (3) 

which shows the respective degrees of freedom to be used 
as divisors in the estimates of variance. 

Example. The entries in the square below are the numbers 
of successes in 26 sets of 10 drawings with probability p ~ 0-52, 
written down consecutively in 6 rows. The mean squares of 
the analyses may be compared with the theoretical which 
is 10 X 0-62 X 0*48 = 2-50. 

The working details, based on the formulae of 75, 76, 77, 
are shown (i) in ordinary row and column analysis, applicable 
equally to h rows and k columns, (ii) in Latin square analysis, 
using the particular Latin square (g.v.) given above. 


(i) 6 

3 

2 

4 

6 

Sums. 

20 

Means. 

4*0 

6 

7 

6 

4 

6 

27 

6*4 

3 

6 

3 

6 

6 

23 

4*6 

8 

3 

7 

6 

4 

28 

6*6 

6 

2 

6 

4 

6 

23 

4*6 

Sums 28 

21 

23 

24 

26 

121 


lieaxiB 6*6 

4*2 

4*6 

4*8 

6*0 


4*84 



SCHEME OF NUMERICAL ANALYSIS 


141 


Sums. Means. 

23 4-6 

22 4*4 

25 5 0 

29 5-8 

22 4-4 

Total . . 121 4*84 

The various sums of squares used are : (1) the sum of 
squares of all 25 entries, namely 647 ; (2) the sum of the five 
products, row-sum by row-mean, 594*2 ; (3) the same for 
columns, 591*0 ; (4) the same for letters in Latin square, 
592*6. Each one of these must be corrected for transference 
to the general mean 4*84, and the correction in every case is 
to subtract the product of total sum by total mean, 121 by 
4*84, or 585*64. 

Thus the corrected sums of squares are (1) 61*36, (2) 8*56, 
(3) 5*36, (4) 6*96. The residual sum of squares is found by 
subtraction from the total sum, and the details of estimate of 


mean square are set out in tabular form thus 

: 

(i) Row and column analysis. 




Sum sq. 

Degr. 

Mean sq. 

Rows 

8*56 

4 

2*14 

Cols. . 

5*36 

4 

1*34 

Res. . 

47*44 

16 

2*97 

Total . 

61*36 

24 

2*56 

(ii) Latin square analysis. 

Sum sq. 

Degr. 

Mean sq. 

Rows 

8*56 

4 

2*14 

Cols. . 

5*36 

4 

1*34 

Letters 

6*96 

4 

1*74 

Res. . 

40*48 

12 

3*37 

Total . 

61*36 

24 

2*56 


We need not continue, but in practice the mean squares 
for rows, columns and letters would be compared with each 
other and with the residual mean square by taking half the 


Latin sq. 

(ii) A 

B 
C 
D 
E 





142 


STATISTICAL COEFFICIENTS 


difference of logarithms and applying the z<test. (The 
logarithms are Napierian, and we may note the relation 
i log^u = 1-161 logioM.) 

The principle of isolation, by appropriate experimental 
design, of the separate variations due to several simul¬ 
taneous causes, has been developed and widely applied 
in recent years. Complex patterns, such as randomized 
blocks in which each element is itself a block, or Latin 
squares in which each ‘‘ letter ” is a Latin square, have 
been designed and used. The idea is to save time, space 
and expense by being able to conduct several kinds of 
experiment at the same time and within the one frame. 
For further details the reader may consult Fisher’s The 
Design of Experiments, 2nd edition, or Yates’s The Design 
and Analysis of Factorial Experiments (Harpenden, 1937). 


78. Conclusion. The consideration of other sampling 
distributions would exceed our space and scope, but one 
of special interest may be noted. The distribution of r, 
the standardized product - moment estimate (without 
Sheppard’s correction) of p in normal correlation, was 
found by R. A. Fisher in 1916. The probability function 
has the rather complicated form 


^(r) = c(l-p2)|(n-l)(2-r2)*(«-4) 




arc cos (—rp)^ 

, V(l-r»P^) /(I) 


and the curve, if p is at all large and the sample small, 
is skew and in cases even U-shaped. (The function and 
its integral have been computed, for w = 3, 4, ..., 25, 50, 
100, 200, 400 and p = 0-l, 0*2, 0-3, ..., 0*9, by F. N. 
David in Tables of the Correlation Coefficient, London, 
1938.) 

It was proved by Fisher (Metron, 1921) that the 
hyperbolic tangent transformation 


*' = i log 





. ( 2 ) 
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produces a distribution which even for n as small as 20 
is nearly normal, with mean J and variance l/(n--3). 

A second transformation of r, namely 




Vn—2, • 


. (3) 


leads to a i-distribution with n—2 degrees of freedom. 
These transformations are necessary, because of the 
extreme non-normality of the sampling distribution of r, 
which makes the crude use of the standard error of r a 
fallacious procedure. 


79. Estimation of Parameters from Sample. In 

40 and 42 we have estimated the mean of a normal and 
a Poisson distribution by the mean m of the sample, in 
43 we have pointed out the demerits of the mean of sample 
in estimating the true mean of a Cauchy distribution. 
The general problem of estimation is this: given n sample 

values .of a variate x with probability function 

<f>(x ; 0) involving a parameter 0, what function T(Xj^, 

..., Xn) of the sample values shall be used to estimate 0 ? 
The problem must be posed m mathematical terms, and 
must, in order to become intelligible, assume a certain 
degree of arbitrariness. One fruitful principle, well 
justified by its results, consists in choosing T by making 
the compound probability density of x^, x^y ..a maximum 
with respect to d. This is R. A. Fisher’s principle of 
maximum likelihood. Another principle postulates (i) that 
T shaD be unbiassed, in the sense that the mean value of 
T over all samples of n data shall be equal to 0, (ii) that of 
all such functions T shall be the one with minimum sampling 
variance. In many cases these two different approaches 
(the second of which has not yet been deeply explored) 
lead to the same function T of estimate. The situation is 
parallel to that wliich occurs in the theory of Least Squares, 
where, as mentioned at the end of 57, different sets of 
postulates lead to the same normal equations. 
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80. Four-Place Table 


X 

❖ 

X 


0-00 

0000 

0-60 

1916 

002 

0080 

0-52 

1985 

0 04 

0160 

0-54 

2054 

0 06 

0239 

0-56 

2123 

0-08 

0319 

0-58 

2190 

010 

0398 

0*60 

2257 

012 

0478 

0-62 

2324 

014 

0557 

0-64 

2389 

016 

0636 

0*66 

2454 

018 

0714 

0*68 

2517 

0*20 

0793 

0*70 

2580 

0*22 

0871 

0*72 

2642 

0-24 

0948 

0*74 

2703 

0-26 

1026 

0*76 

2764 

0-28 

1103 

0*78 

2823 

0*30 

1179 

0-80 

2881 

0*32 

1255 

0*82 

2939 

0*34 

1331 

0*84 

2995 

0-36 

1406 

0*86 

3051 

0-38 

1480 

0*88 

3106 

0*40 

1654 

0*90 

3159 

0*42 

1628 

0*92 

3212 

0-44 

1700 

0*94 

3264 

0-46 

1772 

0-96 

3315 

0-48 

1844 

0*98 

3365 

0-60 

1915 

1*00 

3413 


of $(*) == (2it)-» [e- *^’dx. 

JO 


X 


X 


1*00 

3413 

1*50 

4332 

1*02 

3461 

1*66 

4394 

104 

3508 

1*60 

4452 

106 

3654 

1*65 

4505 

1-08 

3599 

1*70 

4554 

1*10 

3643 

1*75 

4599 

1*12 

3686 

1*80 

4641 

1*14 

3729 

1*85 

4678 

1*16 

3770 

1*90 

4713 

1*18 

.3810 

1*96 

4744 

1*20 

3849 

2*00 

4772 

1*22 

3888 

2*10 

4821 

1*24 

3925 

2*20 

4861 

1*26 

3962 

2*30 

4893 

1*28 

3997 

2*40 

4918 

1*30 

4032 

2*50 

4938 

1*32 

4066 

2*60 

4953 

1*34 

4099 

2*70 

4965 

1*36 

4131 

2*80 

4974 

1*38 

4162 

2*90 

4981 

1*40 

4192 

3*00 

49866 

1*42 

4222 

3*20 

49931 

1*44 

4251 

3*40 

49966 

1*46 

4279 

3*60 

49984 

1*48 

4306 

3*80 

49993 

1*50 

4332 

4*00 

49997 


A decimal point is understood before each entry ^(a?) ; 
and second difference interpolation is advisable in the last 
column. 

A useful inverse table of the normal probability iutegral 
is the table of Probits in Fisher and Yates’s Statistical Tables 
for Biological, Agricvlturol and Medical Research (Oliver and 
Boyd, 1938), pp. 38-40. The “ probit ” is the value of a; which 
cuts off at its ordinate a given percentage of area measured 
from the left of the normal curve. 
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1. Finite Differenoee and Factorial Polynomials. Most 

tables of functions provide us wiih sequences of values which 
by a suitable choice of origin and scale may be denoted by 
tip, u^, u,,.... To these we may apply differencing and repeated 
differencing, analogous in the Calculus of Finite Differences 
to differentiation in the Infinitesimal Calculus (see Whittaker 
and Robinson, Calculus of Observations (Blackie), Chapters I 
to IV). The operations most commonly used are : 

the advancing difference, Aug. == Wa> 4 -t — t/,, 
the receding difference, “ t/, - 

the central difference, Sw* = —• (1) 

the averaging operation, fiUg. — 
the moan central difference ^ 

operations which may all be repeated. The classical formula 
of interpolation, which uses advancing differences derived 
from Uq, is the Gregory-Newton formula 

u, = + • (2) 

a formula which terminates at n-(-1 terms if is a polynomial 
of degree n, €Uid which in practical cases converges well, 
with negligible remainder, after a few terms. The formula 
is the analogue of the Taylor series in the Infinitesimal Calculus. 

The polynomials 1, x, ./<*>, ... which appear in (2) 
are (29) ordinary factorial polynomials 1, *, x(a; —1), 

x(x — \)(x — 2), .... If they are divided respectively by 01, 
1!, 2!, 3!, ... we obtain the reduced factorials or binomial 
coefheients 1, a;, ... . 

Central factorials may be defined by 

1, 3 ^^} = aK*) r= (a;-f J)(a?—J), aK*) = (a;-f l)(a-(a? — l), ..., (3) 

the factors being in arithmetical progression of common 
difference unity and centred at x. The render may verify that 
X, =r x*, /io;!*! -- —i), *= a;*(a;* —1), (4) 
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Given an odd number of values of t/, with oentrnl value 
Uq, the Newton-Stirling formula of interpolation is useful, 

Uf, =» + + • (5) 

Given an even number of values with two central values 
and U|, the Newton-Bessel formula is the appropriate one, 

These formulas use central and moan central factorials, and 
mean central and central differences, alternately. 

The origin of interpolation x = 0 can almost always be 
chosen so that x, in the interpoland m», need not exceed J. 

The following relations are fundamental: 

Jaj(r) ^ equivalent to Sxl*'! = rx^f- U, 

(Cf. Dx^ = in the Differential Calculus.) 

2. Finite Sums. The following table of repeated 
summation upon Wj, ..., exemplified for n = 6, 

follows the scheme proposed in 19 for computing factorial 
moments. 

u E E* E^ 

Wo Wo+Wi-fWj+Wo-fWo 

ttj Wi+Wj-fWj-fWo Ui-f 2 w,-f 3 m,-|-4uo 

W, W,-fU,-fU 4 tt, + 2tt, + 3W4 Wg f 3t/, f 6^4 

Wj Wj-fW 4 U,-f 2 t <4 U, + 3ti4 l/j4 l“W 4 

ti 4 W 4 U 4 M 4 7^4 

Scrutiny will show that the entries at the tops of the 
successive columns of summation are the reduced factorial 
moments: 

EUmt ExUg, EXf^)Ugif EXi^)Uxt Ex^^^M4x9 ••• • 

This may be proved by an induction based on Ax^f) = 

With a little more difficulty, using central factorials, it 
may be proved that the scheme of repeated summation 
toward the centre with alternate averaging, used in Ex. 3 
of 19, produces reduced central and mean central factorial 
moments Exi^^Urjrl and Efisd^hixIrU 
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3. Relations between Powers and Factorials. We have 

(i) X — X, (ii) X — X 

x^ ~ 

a;3 = a;(3>_|_3a5(2)_|_jj.^ 3.3 _ x^^)-\-x, 

x^ — x^*^-\-7x^^^-\’X, x^ fxx^^^-{-fjiX^^}. 

as may be verified by actual expansion. Multiplying any of 
these relations by te* and summing over equally spaced values 
of X, (i) with a? = 0 as least value, (ii) with a: = 0 as middle 
value, we derive the relations quoted and used in 19, Exs. 1 
and 3, for converting factorial moments, or central and mean 
central factorial moments, into ordinary moments. 

4. Tables of Normal Probability Integral and Poisson 
Function. A very convenient table of the normal probability 
integral in standard scale, to four places of decimals, is given 
in Bowley’s Elements of Statistics, p. 271. The table is 
accurate enough for most practical purposes, and may be 
interpolated by proportional parts, that is, only using first 
differences. We give a compact table in 80, p. 144. 

In the Poisson function the chief requirement is the value 
of If a machine is available, the following short table 

enables e-”* to be computed with sufficient accuracy for 
m = 0 to 10. 

m e-^ m e~”* m e~”* m 

1 0-36788 0-1 0-90484 0-01 0-99005 0-001 0-99900 

2 0-13534 0-2 0-81873 0-02 0-98020 0-002 0-99800 

3 0-049787 0-3 0-74082 0-03 0-97045 0-003 0-99700 

4 0-018316 0-4 0-67032 0-04 0-96079 0-004 0-99601 

5 0-0067379 0-6 0-60653 0-05 0-95123 0-005 0-99501 

6 0-0024788 0-6 0-54881 0-06 0-94176 0-006 0-99402 

7 0-0009119 0-7 0-49659 0-07 0-93239 0-007 0-99302 

8 0-0003355 0-8 0-44933 0-08 0-92312 0-008 0-99203 

9 0-0001234 0-9 0-40657 0-09 0-91393 0-009 0-99104 

10 0-0000454 1-0 0-36788 0-10 0-90484 0-010 0-99005 

For smaller values of m than those given above the approxi¬ 
mation 1—m for e-"* is correct to at least five decimals. 

Ex, In the example of 42 we have m — 3-870. Entering 
the above table at m = 3, 0-8, 0-07, we form the product, 
thus : 0-049787 X 0-44933 X 0-93239 = 0-20858. 

k2 
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6. Linear Dependence, Functional Dependence, Cor¬ 
relation, Statistical Dependence. These are concepts 
which need careful discrimination. The functions Uf(x)f 
where / = 1, 2, n, are linearly dependent if a relation 

= 0, . . . (1) 

exists identically in x, where one at least of the is not 
zero. They are functionally dependent if a functional relation 

M,) = 0 . . . (2) 

exists identically in a?. Linear dependence for example, is 
the case where F is a non-zero linear function. They are 
uncorrelated it the product moment /nn vanishes for each pair 
Ui and Ilf of the set. 

Correlation and functional dependence are (48) not 
necessarily the same. The simplest example is perhaps 
u = a cos x-{-b sin a;, v == a sin x—b cos x. Here u and v are 
uncorrelated, yet are dependent in view of the quadratic 
relation 6^ = o. 

To describe statistical dependence, we may say that 
statistical independence is really obedience to the multiplica¬ 
tion theorem of probability. Suppose we have two functions 
of n variates, u(x, y, z) and v(x, y, z), where we illustrate by 
n = 3. They have each a probability, or probability density, 
let us say tpiiu) and ^ 2 (^)» depending on the distribution of 
X, y and z. They have also a compound probability, or 
probability density, let us say ^{u, v). If for all the possible 
values of x, y, z we have v) = ^i(w) 02 (v)» t^^en we say 
that u and v are statistically independent. 

An equivalent formulation is by generating functions. If 

0{a, P) — j j je^'^~^P*’<j,(x, y, z)dxdydz . . (3) 

where <l>{x, y,z) is the compound probability density of x, y, z, 
and if 0(a, P) = 0{a, 0)0(0, P), . . . (4) 

all integrals existing in some common domain of a, p, then 
u and V are statistically independent. By this criterion it 
may be proved that the estimates m of y, and s^ of a* in a 
normal sample (72) of n values o^^are statistically independent, 
so that the derivation of the ^-distribution (loc, cit.) is valid. 
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Helmert, 130 
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Lerch, 128 
Lexis, 62, 100 
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Authors cited— 

Pearson, K., 37, 67-69, 71, 
72, 95, 100, 101, 104. 128 
Poisson, 50, 51, 52, 59 
Rutherford and Geiger, 77, 
78, 103 

Shoppnrd, 39, 44-47, 73, 76, 94 
Tchobychef, 115 
Wald. 9 

Whittaker and Robinson, 122, 

124, 145 

Yates, 105, 142, 144 
Yule, 25, 106 
Yule and Kendall, 25 
Averages, 29, 30 
x4xiomatization, 3, 4 

Bcrnoullian variance, 49, 50-54 
Beta function, 71, 72 
Binomial distribution, 49, 58, 

125, 133 

approximations to, 69-61, 63 
of Poisson, 50, 51, 58 
Binomial correlation, 82, 83 
Bivariate distribution, 80, 86, 
94, 95 

generating function, 83, 84 
Blocks, randomized, 138, 140, 
141, 142 

Central factorials, 145, 146 
Central factorial moments, 43, 
44, 146, 147 

Change of origin and scale, 23, 60 
Change of variate, 69, 135, 136, 
142 

Classification, 2, 5 
Coefficient of correlation. 86-87, 
90-93, 113-114 
of perturbation, 55 
regression, 112 
Complementary event, 13 
probability, 13 

Compound probability, 14, 15 
Computation of moments, 39-43 
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Contingency table, 82, 83, 99, 
100, 102, 104, 105 
Corrections, Sheppard’s, 39, 44- 
47, 73, 76, 94, 142 
Correlation, 80-103, 111, 112, 
113, 114, 148 
binomial, 82, 83 
coeificient of, 86, 87, 90-93, 
112-114 

hypergeometric, 84 
non-linear, 95-98 
non-metrical, 99-105 
partial, 113, 114 
Poissonian, 94, 95 
ratio, 95-99 
surface, 82, 88 
table, 89-93 
total, 112, 113, 114 
Covariance, 84 

Criteria of homogeneity, 54, 55 
Cumulant, 22, 23, 32, 61, 64, 
65, 70, 128 
factorial, 23, 64 
generating function, 22, 64, 
65, 67, 70 

Degrees of freedom, 102, 103, 
123, 130, 131, 133, 134, 
137-140, 143 

Density, probability, 16, 143 
Dependence, linear, 101, 102, 
103, 130, 148 
fimctional, 88, 148 
statistical, 13, 14, 15, 87, 88, 
102, 130, 148 

Dependent events, 15, 16, 56 
Deviation, mean absolute, 32 
standard, 35, 37 
Difference of means, 134 
Differences, finite, 59, 67, 115, 
118, 119, 145, 146 
Dispersion, 32, 34, 35 
residual, 96 
Distribution— 

binomial, 49, 58, 125, 133 
binomial of Poisson, 50, 61, 58 
bivariate, 80, 86, 94, 95 
Coolidge, 53-55 
frequency, 26 

Gamma t 3 ^, 69, 72, 102. 128 
Hehnert’s 130 


Distribution— 

hypergeometric, 66, 67 
J-shaped, 27, 64 
leptol^rtic, 38 
Lexian, 53, 54, 55, 72 
multinomial, 65, 101 
multivariate, 80, 101 
normal, 58-62, 72, 73, 74 
normal correlated, 86-89, 101, 
113 

of Fisher’s z, 135, 136, 138 
of r, 92, 142, 143 
of Student’s /, 131-134, 143, 
148 

of sum of squares, 69, 129 

of X*, 101, 102 

of variance estimate, 130-131 
Pearsonian system, 67-71 
platykurtic, 38, 71, 127. 133 
Poisson, 68, 59, 63, 64, 66, 
77, 78, 103 

Poisson correlated, 94, 95 
probability, 26 
rectangular, 48, 79 
sampling, 92, 126-127, 133 
skew, 27, 31, 36, 37, 58, 69, 
72, 127, 131 
synmietrical, 27, 61 
trivariate, 80, 112, 113 
Type A, 58, 59, 64, 65, 66, 
67, 73, 75, 76 

Typo B, 68, 69, 66, 67, 73, 
76, 77, 78 
Type I, 71, 72 

Type III, 69, 72, 102, 128, 129, 
130, 131 

U-shaped, 27, 69, 142 
Dot diagram, 80 


Ellipse, probable, 88 
Empirical formula for P(y*), 
104 

Equal likeliness, 10, 11 
Equations, normal, 110, 114, 
115, 116, 121 

Error function, 62, 73, 74, 76, 
144, 147 

Error of mean, 128 
of moments, 39, 126 
of r. 92, 143 
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Error of sampling, 39, 92, 126- 
129, 131, 143 

of variance estimate, 129-131 
probable, 35 

standard, 39, 92, 126-129, 131, 
143 

Errors and residuals, 107, 108 
Estimation from sample, 78, 79, 
143 

Euler-Maclaurin formula, 44 
Events, 5, 6 

dependent, 13, 15 
independent, 13, 14, 15, 18 
mutually exclusive, 13, 18 
Excess, 38, 63, 131 
Expectation, mathematical, 21 

Factorial moments, 21, 22, 41, 
49. 63, 84, 85, 146, 147 
moment generating function, 
22, 49, 63, 84, 85 
polynomials, 21, 145, 146 
cumulants, 23, 64 
cumulant generating function, 
64 

Factorials and powers, 147 
Factorials, central, 145, 146 
Finite differences, 59, 67, 115, 
118, 119, 145, 146 
sums, 40-43, 146 
Fitting of harmonic function, 
120-123 

of polynomial, 114-120 
of probability curves, 73-78, 
143 

Formulie of interpolation, 145, 
146 

Fourfold table, 82, 85, 94, 95 
Fourier, transform, 22 
Frequency, marginal, 82, 100, 
102 

relative, 4, 5, 7, 26, 60 
Frequency polygon, 28 
Function, probability, see Dis¬ 
tribution 

Functional dependence, 88, 148 

Gamma Type, see Distribution 
Generating fimction, 15, 16, 17, 
19, 148 

bivariate, 83, 84 


Generating function, change of 
origin and scale in, 23 
factorial moment, 22, 49, 63 
84, 85 

factorial cumulant, 64 
moment, 20-24, 60, 65, 69, 
75, 84, 85, 86, 101, 113 
multiplication theorem, 19, 22 
cumulant, 22, 64, 65, 67, 70 
Goodness of fit, 76, 78, 100-103, 
104 

Gregory-Newton formula, 145 

Harmonic regression, 81, 120-123 
Helmort’s distribution, 130 
Histogram, 28 

Homogeneity, criteria of, 54, 55 
Hypergcometric correlation, 84 
distribution, 56, 57 

Independence, functional, 88, 
148 

linear, 101, 148 
statistical, 87, 148 
Independent events, 13, 14, 15, 
18 

frequencies, 102, 103 
Inductive synthesis, 3 
Integral, probability, 62, 72, 73, 
133, 144, 147 

Interpolation formula?, 145, 146 

J-shaped curve, 27, 64 

Kurtosis, 38, 63, 131, 133 

Latin square, 139-142 
Least squares, 95, 106-108, 110, 
111 

Leptokiirtic, 38 
Lexian ratio, 55 
Lexian variance, 53, 54, 72 
Likelihood, maximum, 143 
Likeliness, equal, 10, 11 
Limit of relative frequency, 7, 8 
Limits of r and p, 87 
Linear dependence, 101, 102, 
103, 130, 148 

Linear function, of moments, 
36. 37 
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Linear regression, 88, 89, 106, 
112 

Logic, algebra of, 6 

Maximum likelihood, 143 
Meem absolute deviation, 32 
Mean central factorial, 145 
Mean, median and mode, 30, 31, 
37 

Mean square contingency, 104- 
105 

Mean square, distribution of, 129 
Measure of aggregate, 10 
Median, 30, 32-34 
Moments, 20, 31, 37, 38 
computation of, 39-43 
see Factorial moments. Gener¬ 
ating functions 

Minimum variance, principle of, 
143 

Multinomial distribution, 55,101 
Multiplication theorem, 14, 15, 
19 

Mutually exclusive events, 13,18 

Non-linear regression, 95, 114 
Non-metrical correlation, 99-105 
Normal curve, see Distribution 
Normal equations, 110, 114-116, 
121 

Optimal values, 108, 109 
Origin, change of, 23, 29 
Orthogonal functions, 116, 120, 
124 

polynomials, 116, 116, 120, 
124 

Parameters, estimate of, 78, 79, 
143 

Partial correlation, 113, 114 
Pearson curves, 67-71 
Pearsonian coefficient r, 86, 87, 
90-93 

Periodic regression, 81, 120-123 
Perturbation, coefficient of, 66 
Phase aggregate, 10, 12, 14 
Platykurtic, 38, 71, 127, 133 
Poisson binomial, see Distribu¬ 
tion 


Polynomial, factorial, 21, 146, 

146 

Polynomial regression, 81, 114, 
116-120 

Population, 24, 25 
Powers and factorials, 147 
Precision, 107, 108 
Preparation of normal equations, 
110 

Prismogram, 81 
Probability, 4-12 
d priori, 6, 9 

as limit of relative frequency, 
7, 8 

as measure of sub-aggregate, 

10, 12 

complementary, 13 
continuous, 12 
curve, 27 
definition, 6, 9, 12 
density, 16 

distribution, see Distribution 
function, 16 

fundamental theorems, 13, 14, 
15 

integral, 62, 72, 73, 133, 144, 

147 

marginal, 82, 100, 102 
of dependent events, 16 
parameters, 29, 30 
polygon, 28 
total, 13, 82 
Probable error, 36 
Product-moment, 84, 86, 87, 90 
Provisional mean, 39 

Quartiles, 34, 36, 37 

Randomized blocks, 138, 140, 
142 

Randomness, 9 
Range, 35, 36 
Ratio, correlation, 96-99 
Lexian, 65 
of variates, 135 
Student’s, 131, 134, 143, 148 
Rectangular distribution, 48, 79 
Regression, 80-82, 88, 89, 95, 
106, 112-124 
coefficients, 112 
lines and planes, 88-89,106,112 
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Relative frequency, 4, 6, 7, 26, 
60 

Replacement, sampling without, 
56 

Residual, 108 
dispersion, 96 
variance, 109, 119, 123 


Sample, 24, 25, 86, 107, 125-136 
estimation from, 78, 79, 143 
Sampling distribution, 92, 125- 
135, 143, 148 

error, 39, 92, 126-135, 143 
of r, 92, 143 

Sampling without replacement, 
66 

Science, pure and applied, 2, 3 
Seminvariants, 22, 23 
Semi-intorquartilo range, 35 
Series of Type A, B, see Dis¬ 
tribution 

Sheppard’s corrections, 39, 44- 
47, 73, 76, 94, 142 
Skewness, 27, 31, 36, 37, 68, 
69, 72, 127, 131 
Square, Latin, 139-142 
Standard deviation, 35, 37 
error, see Error of sampling 
Statistical dependence, 143 
Statistics, definition, 1, 5, 7, 
12 

Student’s f, see Distribution 
Sum of squares, analysis of, 54, 
136-110 

distribution of, 69, 129 
Summation method for moments, 
40-43, 146 
Symmetry, 27 
Synthesis, inductive, 3 


Tables, British Association, 75 
contingency, 82, 83, 99, 100, 
102, 104, 105 
correlation, 89-93 
fourfold, 82, 85, 94, 95 
of Fisher’s z, 136 
of Poisson function, 147 
of P(x^), 103, 105 
of probability integral, 73, 
144, 147 

of Student’s t, 133, 134 
of terms in Type A, 75 
Tabulation, 1, 2 

Tchebychef polynomials, 115, 
117, 119, 121, 124 
Transform, Fourier, 22 
Trivariate problem, 80, 113, 114 

Universal, universe, 24 
U-shaped curve, 27, 69, 142 

Variance, 35 

analysis of, 54, 136-140 
Bernoullian, Poissonian and 
Lexian, 51-55, 72 
distribution of estimate of, 
130-131 

minimum, principle of, 143 
of linear function, 36 
of optimal value, 109 
of residuals, 109, 119, 123 
Variate, 16 

additive, 19, 64 
change of, 69, 135, 136 

Weight, 107, 108 

of arithmetic mean, 109 
Weighted mean, 109 

2 -distribution, 135, 136 
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